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(543 Processing of image obtained by multi-eye earners? 



(57) r r _ (. w j clcseci ^ bine iar cd'nera which 
can re a s pane irn viwanc c pic view dur- 
ing image sensing. There is also disclosed a binocular 
camera which has two image sensing optica? systems, 



a circuit for synthesizing right and left sensed paraiiax 
-dimen- 

ssonai image, and a display lor displaying the synthe- 
sized image signal. 
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Description 

pig p esent nve - t - is rg apOc<™ ys mc > at q muRi-eye ca es three 

dimensional nages and/or two-dimensional images of an ofc , nd, moms particularly 

fi i ig panoramic \ ,v and/or s ecscopie view c images obtained bj the imaq c> it 

or three-dimen&icnaisy display moving images with high image quality and high resolution. 

4s<iOO' > - - t-- i , s t ?e-c m i s 5 a p i 5 p dimensional 

dear a r ate _a~ dV-r J „ „ d-c srf-cv r sun* - go sensing disp'a^ 
s> ^ t i I images. => a! a pair of images nawngparaiiaxara omamed from a plurality of cameras, 

and are displayed cn a stereoscopic display dedicafad to I - system, mas presenting a K - ^ a image to a 
user. 

mem tr [ t i 5 n [_ i i trr or; a stereoscopic 

display for displaying a three-dimensional imago are separated, the user cannot stereoseepicaiiy observe the sensed 
images during image sensing and, hence, it *s difficult to adjust Ida cameras So obtain an appropriate f n i 

image white observing the monitor image White sousing an image by moving me cameras, the sie;sosoopic display 
must be disconnected. and ; then, editing of the sensed images by stereoseepicaiiy displaying them is required after 
image sensing. For this reason, image sensing of three-dimensional images is no! easy. m v , it :s hard to 
observe a three-dimensional image by a simple method, either. 

Tine conventional ihrae-dirnensional image system does not consider any r m i vvitn two-dimensional < t 
ages which a p tmlar ^ is ye set rigs ems '~ r sp II twen mail smcs thrs« and 
dimensional image systems are discrete and Independent, if one who has a two-dimensional image system wants to 
sense a m > Image, ha or she mast build a three-dimensional image system. resulting in a heavy load 

on the person. Also, data of a three-dimensional image, computer graphics image, and the like created on a PC cannot 
be displayed on the camera side. 

The p'fe i t t i i t i i n panoramic view and three-dimensional 

image sensing/display for stereoscopic view. Problems posed when moving images are sensed and displayed by a 

ti) system «, >e ciscusse* eiw 

a i r i i a ^ ) v -i 1 or 

panoramic view and the three-dimensional image sensing/display method for stereoscopic view. 

in the Image sensing/Image display method for panoramic view, an image is sensed via two. sight and left image 
sensing opticai systems, which are set so that their view points match each other using mirrors and the like. These 
in i i o ~3 d -t " f " f ; f ^ > t r f c^rip - g o A 

e in I pa i -m _ _ ii m - *o s ^ t - n; „r c m elm n ^ tm mat their 

avsiiappmg regions overlap each cine' o the formed image is hispiaysd on an image output apparaius i>< i as a 
disc-lay 

On the other hand, in the three-dimensional image sensing/image display method for stereoscopic view, two image 
n eg -mr -j t i ii r - r m e mm> 

from two view points Since the average distance between the right and left eyes of a human demo is about S5 mm, it 
is a common practice to set the base distance between the two image sensing optical systems at 65 mm in three- 
dimensional Image sensing/Image display for stereoscopic view. 

When an image of the object o ; interest is sensed t'rom the two right and isn't view points, the object has different 
positions m images sensed by the individual image sensing systems Thai is this difference is parallax, and whan 
images n g parallax m be referred to as "parallax images" hereinafter; am stmaosoop;ca;iv viewed, tne user can 
observe an image witn sediment stereoscopic expression. 

A ^ h d ere caiiy viewing parallax blamed - jht and teft view points tr udas various 

methods. 

One method is a shutter switching methoo n which the obtained parallax images tor the right and left eyes are 

^ f ^ t, f_ — t s Hi p| \ o 

nag ws 1 ssroesewes - > m tauic t r J < r . gf , ana left 

shutters which can be Independently switched, in tne spectacles, since tne shatters are switched in synchronism with 
the display switching timings of the right ano led parallax images, ins user can observe an Image with sufficient ster- 
5 > om expression 

Another display method is ca soacr i nod in this met 02 ?, st ps-pattem ima 

tematsly arranging horl.ronta! lines of two. right and left parallax images Is displayed on. a display apparatus having a 
polarization plate in which trie direction of polarization changes every other horizontal lines The pitch of lines of the 
v 1 o- , a a - ec^s *o the l>ne pftcr of tre s v oe pa 'e ^ pa atl-o> inMQe Wt cn :>■ » st ipe patte' N pa r a > 
is displayed on the display apparatus, the polarization plate transmits only polarized light in one direction coming from 
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the parallax image for the n; eye sensed by the fight image sensing optical system and only polarized light n a 
direction, different from ths direction of polarization of t ngb! image, coming from ths parallax image for the "i eye 
sensed by the eftirna e : ca system. 

On the - ece portions of which have a 

; < - < t,ng ch e -a- it <> these coming from the id p lax 

rr<i ^$ tiisp yed on ths displ -i he specta nit only pel ht tn udrng th^ rght 

oarallax imag ft parallax i» ths Isft ays. With t 

cl - 1 spectacles, when the user observes the right parariax image wrih f right eye atone and ths left parariax 
ra-r At-- -r. rrv a } ti o p 

rc As described above, rhe three-dimensional ■mage sensing/image display method for stereoscopic view uses the 

oar^a* m m i - t ■* r at ~- > pu f n ' - 1 ^ r 3 <n j.rqcaaiax 

so that ,rn 3t ^ e f o c rt ~ „a sfe-ed o 31 i - r e r a. -r t l - 

-ap^o/- 1 --to cm is t <•, are fused, tn sxp ■ u.e. epic s ression 

t-igerv-J r n = a> e-L t y I «.o <~ nd left 

is v t c h - t ~ t m t nh a c ht= 

-a 1 a . j o -g r ti , - a d < - r 1 - 

Hence, the Image sensing optical systems need be a^t *~ - - > ^ w- - - * *■ 0 1 h, 1 

y a iy t is jfobiam is solved by: 

20 {]) setting the image sensing optical systems to have a certain convergence angle, Of 

- al - f ^ - 1 st^ns 

Fig, 1 shows a case wherem the rwo image sensing optica: systems are set to have no convergence angle, l.e , 
are set para ilt f ten other, in Fig. 1 two Image sensing optical systems 6701a and 6?01b are arranged parafiei to 
ZS rsv 5 ' 

6702a and 6702b, and CCDs 5703a and S703b as image sensing devices. Let v Pe the intervs; between the tens 670,2a 
CD 6703a tndthe en/si between (he Sens 6702b and the CCD 5703b, 
Also, assume that a 1 a, object 8904 is present at a position A separated by z from the origin G, in the image 
sensing direction, 

30 in Fig. 1, images of the principal object 5904 are respectively formed on the surfaces of the CCDs 6703a and 
67G : 3b, Ths distance obtained by doodling the deviation of the imaging position ot the principal object 6904 from. the 
CCD center on each of the surfaces at the CCDs 6703a and 87Q3b s ssiicr on a vMh-' st . wop ra e - ge 
aisle ptc systems 6701a b form In sec - 5904 to e given parariax d Corwen 

tionaBy, the image sensing optical systems 6701 a and 3701 b are set to have a certain ogle so that the 

3S c^' N 4, 

A convergence angle 0 can he calculated by equation (1) below on ths basis 0! angles G, A3 and Co AC defined 
bv the centers 6 and C of the Senses 6702a and 57020, trie existing position A of the principal object S904, and the 
origin O, In Fig. 1 



40 




whs /is the distance between each of ihe pa r - tha principal 

ofaleoi 8904, and i is the base distance of the two image sensing optica! systems 3701 a and 6701 b. 
■is w\ n age sensing oc Is 670 la and 010 are relate ngle S given by equ tion I 1 tc 

h c er 3 h<_ f = i 

1 - - - c -Lac - t • - ; ' ™ ~ O^Odb 

rpn J r - -t <• ^ q*m 

angte he para kc 0 the or rscip 3 eb eci e a- ce s» T a C unless there is no ph sic- n ita on s g the image sensing 
so systems S - anc - % 1 --m h ' 

Or >-« om : * - , h 1 " r n >' ms c , " 3 

andc' ' l cm r , d cs ■= - 1 =t - e - ' . --n ^ oc"Cb>. h f-s-h h>c«r^. 2"g 

lenses S7C id 6702b ;o reduce parail t age sensing sy r - 3 and c 

Fig. 3 showe the layot n t - a.-o : 0 V by - tr 

sa sensing oj: svste ' 570 I , ' ' r thj r bye pnensig the base distance 0 > J 

1- snsing op teal systems 6^31a s ro 3"31b ca f a"ax of he sensed images can oe 'etrUcso 

F;g 4 shews an \ - p wherein the paralirax d is rsducsd by paraifeily displacing f CCDs 6703a and 6700b in 
the srnage sensing op-rcai systems 670 1 a and 570~ b »v *n -aspect to the corresponding lenses 5702a and 67020 As 
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shew n Fig. 4 t d nan also be reouc-dbv par he e CCDs 8703a and 6703b to the irr gleg 

positions the p n pa bjecf 6 §04 by t e lenses 6702a and 6702b 

v - •=> k t mt r pld "t,t die t-dirt rv ^no h ee d m-m _ n 5 j 

f 5r stsrsoscop w using t - mage sailing .opa a* s ^n / 

processing and display methods for still Images are available, but no processing and display methods corresponding 
to moving images have been accomplished yet in moving Image display, processing and display methods that can 
cr r r h tj"- r . \, ve 

On the other hand, whan the above-mentioned conventional image sensing optica; systems are set to have a 
certain convergence angle, the conjugate plans of each image sensing surface changes 

In Fig. 2, the conjugate plans of the image sensing surfaces changes horn s conjugate clans 6505a of the image 
sensing surface m parallel view to t, of me -cage sensing surfaces sensed hy fire image sensing optica; systems 
j" ^ 1,. M"' " l" C t r t q 

system € .an pi 3905c o tmag ting surface by i rigi mage se 300 s em 
670 lb. Such changes m conjugate plane of the image sensing surface distort the penpnerai image portion except for 
the image of the principal object 8904 located st the confer of each image. As the convergence angle of She image 
sensing op 1 701 C 01 nes larger ar t < 

attain stereoscopic mew, in order to satisfactorily sense and display images tor stereoscopic view, she converaencc 
angle o f the image sensing optical systems 670 0= end 67010 is limited. 

Qn the other hand, in the method of -educing the parallax of the pr Incipei obiect 5P04 bv m m y displacing trio 
image sensing corneal systems 670 la and 670t b. :f rue base distance is shortened, not only the usage oh the principal 

be obtained. 

'h " ft t id j 1 t r , f 

of the image sensing dsvices is required. When right and led images having la; go paraiiax are used, tree moving amscunt 
becomes too large to reduce the parallax to C by paratteily displacing the Image sensing device, and It becomes hard 
to control ihem 



SUMMARY OF THE INVENTION 



it is.- therefore, an object of tns present invention to provide a mui-l-eye ;ame 
rates and an insage procsssing meteod. which can display mrse -dimensional pictures during image sensing, can adjust 
stereoscopic expression of images while sensing the images, and are compos - - c ^s'ena 'mages 

In order to achieve the above object, according to the present invention, there is provided a multi-eye camera 
o ?r ses a p fy of image sensing m 3 r 1 g an obis eornorising 

first syntheses means for synthesizing a plurality of image signals of the object sensed by the plurality of image 
sensing means; and 

display means iisp g an image sig i * by the - e 

' ' " e of the object sensed by the plurality of Image 
sensing means can bs synthesized info a three-dimensional image signal, which can foe displayed curing image seos- 
< m r sine 

A gtocn J asps sent mvantlo nuiti-oya camera further comprises second syn 

thesis r C ms 06 So' sytheucmg tee oice 1 - * 5 g- • ' mi: 01 „ s^- „ m . u i\ ^ ' - ~ ^ 3 „ 
means to 3 two-dimensional image signal, and " -. display means displays ;ne twe-oimsnsionai image signal synthe- 
sized by the second synthesis means. 

For this reason, tha mult i-eye camera of the prssem Invention can have compatibility with tvvo-dimensiomsi image 

m 5«c j d rdisp ayed 

According to one preferred aspect of the present Invention, since the multi-eye camera further comprises first 
rl - a ^ > 0 l [ t 3 m:1 syn e h r , " SjI 

6' 1 f -^nse- y z i "at t, o ~ ipe ss a ^a ca i v s 7 - n ~i i 

oraEwo-d-em < , 5 < o 3 red oy the operator. 

image signals to a three-dimensional image signal, and a display period c: the three-dimensional image signth on the 
- nns ans is set t g n goal 

According to one preferred aspect of the present invention, the display means comprises a detachable lenticular 
lens on a dismay surface thereof. 
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A' to one preferred aspect o? the present invention, tre display means comprises a detachable parallax 

barrier between itself and aft- observer. 

According e f da he present rvertton, me <~<uti eye carre a u ises an strip 

t 1 " " " *C C ^ T 

period rJ nsi nat. ~o - or a i 3 by the operate 

can be displayed 

According to one preferred aspect of the present invention, the rnuiti-eye camera further comprises second seiec- 
>: means - ne of the plurality cf ^ma "~ 3 3gs 

tm c- s r ~c r -r t „ sms -I i - - ' t f i a r - \r a ^ 

be sensed. 

According tc cos prefaced aspect of the present invention, trie plurality of image sensing means ere held to bs 
pivotal with respec o the display means, and s r angle neansfor 

r> u I / f s ^ ir 

- r appropriate m ner mage or tv i i i xidencs * tht e pivot 

angle. 

According to one profaned aspect of fee present invention, she multi-eye camera turtner comprises third selection 
means for alternatively selecting the second and first synthesis means on the basis of the pivot angles detected by the 
^ e * - '- - -n r c l. t ee - it. o m " \" 

dimensional image ;r - ^ p - a c 

tn erser to achieve the above object, according to tne present invention, there Is provided an imago processing 
method comprising: 

he im ef sensing an image of an 

be nif In it ;n r r e - top 

, <• ree dimensional — r r i if.it 

t t a gr it ns!ng step can be syn esizt into a inreeariimensiot ftr e 

, a 1 v i i c, t .1 *ud doing page sensing, thus allowing the operator to adjust stereoscopic 

stern of synthesizing the plurality of image signals of the object sensed =n the image sensing step to a two-dimensional 
image signal, and tne display step netedes tne s;ep of displaying the two -d inn en s ton a i image signal syntnesizod in the 
second synthesis step The method of the present invention has compatibility wish two-dimensional Images that are 
;u- re / most frequent d, and not * din " 3 r. a to. sional image sigr-a 

can be synthesized and oisptsyed. 

According to one preferred aspect ot the present invention, the method further comprises the first selection step 
of i <^r<- t It H r * ~ u p u al ty Ot i^age s ana s of ns oqem a-msed in 

the image sensing step can be synthesized into a three-dimensional image signal or two-dimensional image signal 

r-=s r\ t s epe - - 

ye \ e > i-nc *i m t ; p' i'Os t rruit o, s - m" a *s *^'e— - ,.ystem 
which allows one to always observe a three-dimensional picture during image sensing, to adjust stereoscopic exptes- 
i- 1 krn • P n] nom- 

patib&ty wild conventional two-dimensional pictures ana high affinity with a PC system, and is easy to operate, 

c r g oh mven n ed an image sensing ap 

paratus comprising: 

8 plurality of image sensing means, and 

jtpuf means to; dutptifting a a f im ed by the plurality of im; i 

synchronization signal synchronized with each of the plurality cf image signals. 

The apparatus can sense three-dimensional images, and send them to an external display to display a three- 

According to one preferred aspect of the present invention tne apparatus runner comprises display means for 
display^ the plurality cf image signals so as to beahls fo bs observed by both eyes in synchronism With each of the 
or . a . 1 cscms*c ::' ---us-:: •••• „ s the camera 

itseif. 

According to one preferred assect of the present invention, the Image signal output from the output means is a 
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standard television signs!. A conventional television system can be used. as the- externa! display. 

According to one preferred aspect of the rcsen! -vennon the image signs! output from iMe output means is an 
y roans of a compi I T j pc- atus car sense three-din f 
tm g s it d -eno I • r m ~ re " J thx JO r 

5 e « t r - i 3 t cr t t - i h i \ ' 

image signals on a time axis. A - I image can be transmitted to an externa; display by alternately cut- 

pacing the ! and left ages ^ rj M f >:is 

A - nepf fer j aspect of resent mv tne output moans outputs the plurality a? m sigr is 

so that individual image? of the plurality of image signals am arranged on a display screen ot r - (. i i display 
io rnt =■ a t a i t h , b - -< m <m a r -"- ^ . -t a ■> j 

putting the right and left images which are sat spafis!Ty> 

According to one prefansd aspect at tne present invention, the apparatus further comprises' 

mode setting means tor selectively setting one of a thre 
is playing the plurality or image signals, and a two-dimensi 

8 rsnais anc two d rmm nallv 0 r 



orooessed signal t> 



;iion rhaans for 
connected. 



2S a p >.J*y v,-j ; fA mc- m 

input means t i insetting a ihreeofimensicnai "megs signal from an external device; sne 

s ^ r I i 1 f I / v. 

npu cji i v I imer - ined by the n ; r s 

30 With this apparatus, i ~ t m image signals creased by en externa! device can be input to and displayed 

on the camera.. 

r( l<ag to one preterm - m es ght anc 

t / c a ^;e os - 1 ic - is^L^i-npcrt- h f- 

According to one preferred aspect ot the present invention, the t <- ~ m r image s-gna! is script data for 
35 g=f|.se rj . > - - ~ i ~ f A three-dimensional image can be created inside the camera on the basis of 

the scrip; data sen! from. e.g.. a computer. 

AccQ'o j to one preferred ^ 1 t present invention, the plurality c mage input means respectively comprise 
n v so i r :jr - t i - > - k x r. -~- r - p s th p 

dimensional image signals created by an external device can be realized. 
40 A t ! ideality of Image input mean 

comprises an adapter device tor inputting a threo-dmsmsiona! image signal. 

3 ;o the rs rlion r u i r r r put m 

Gutputfing the plurality ot image signals input from the plurality of image input means, and a synchronization signal 
h on « f "* t f h f l t i - J 1 

45 receive tnree-dimensmnai inroge data created by an externa! device. Also, the camera can output the sensed three- 
dimensional images to an external device together -mm synchronoahon signaie. 

it is stsil another object of the present invention to provide a multi-eye image sensing merhod and apparatus, which 
can implement mags t r „ a for panoramic view and -< o s ..< - a j 1 sensing/image display for ster- 
eoscope view ot moving images m correspondanca whir the output format 
so r m hove "he aocve object jeco ^cdo ti <, - r 

comprising: 

t image syn esis m f j s-nd 

the step of synthesizing two, right and left images sensed by two. right and iett image sensing optical systems to 
ss a s ngje sym> 7 f ^ ssco dance a ti i^e ' mm .s-^-vx 

in this manner, the image sensing/display for panoramic vlewand t A n r image censing/image display 
osc i, .• ^ ■ . ~ " .. s for [ t 
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According to one prefer; ed aspect of ths present invention, the plurality o; synthesis methods Include 3 firs; syn- 
thesis method for synthesizing the images while giving priority to a synthesis spaed, and a second synthesis method 
for syrrtfrssizir ted Image 

According to one preferred asp-set of the present invention, the first synthesis method synthesizes the two. right 
and left sensed images by giving a predetermined overlapping amount, and 

she second synthesis method corrects right-and-left differences of luminance levels and cofor information, and 
t dtst t " ges. detects an n ^ r <t vc 

and synthesizes the two images us c = - c a s-cir g ame it jiatea based c he e ping eg ion 

Wr ir r l. t f ^ ^ - u f selecting th 

t <- ^ F r i - - rt- - no np 

due a on modes 

^cording to one prefer re p : 3 nt if r nthesized image :s a p ttuc synthesized 

image. 

According to one preferred aspect oi the present invention, the synthesized .megs is an image for stereoscopic 

Inofdt to -a is at eject ding to t p t vsntx the s provided an image sensing ap 

paralus comprising: 

synthase iuar s ha vnca P u -h , ~ " ,vo t unit 

it-, r ' r 3 - , r -i 

switching means for switching the plurality oi synthesis methods 

According ;o one preferred aspect of the present invention, ice plurality of synthesis methods include a fire! syn- 
thesis ^ <r hod u' or < - - P h i , * v i '"h s i 
for synthesizing images while giving priority to Image quality of the synthesized image. 

According to one preferred aspect of the present invention, the firs; synthesis method synthesizes the two, right 
and left sensed images by giving a predetermined overlapping amount, and the second synthesis method corrects, 
right-and-left differences o? luminance levels and color information, and trapezoidal distortions of the two. right and left 
sensed images, deiec a verlapt 3 betv s two images, and nfftesizes the two in ages using in 
overlapping amount calculated based on the overlapping, region. 

According to one preferred aspect of the pr esent invention, the switching means selects the first synthesis method 
in a through display mode, and selects the second synthesis method in recording and reproduction modes. 

It Is sti another oblec of the present invention to provide molti-eye nag sensing method and apparatus which 
can obtain an Image for stereoscope view, which allows easy fusion of images of a principal rut 

In order to achieve the above object, according to me present rvertbn. there is provided an image sensing method 
comprising: 

the step of sensing a pair of mages having parallax using two image sensing optical systems: and 

ths control step ot controlling to adjust parallax of a pair of partial images of a principal ooiect selected from the 

pair of sensed images. 

Rmthfs ^ - - - ■> ^ f < ' ~\ . - ^g&s ■ ht ms p - - oan^s 

obtained. 

According to one preferred aspect of the present invention, the - \ step includes J step of setting a i n \ value 
of a conversance angle of each of the image sensing opticai systems. 

According to one preferred aspect of the present invention, the control step includes the step of -p > the 
pataiiax o! tne principal object in the images by paraiieiiy displacing an Image to be c , to when trie -n - - • v 
5t.U£f ,, l ^" " = " ^ " "- sh - to r 

itecording rse starred asp he present* sntion. the paraiiel-dispiacema : see e image 

senstn - e ^ * <~ st - j c j , u to ~ 

- d - i n , s "p : ; . r r r- r s 

gence ang ea rnag nsing sc.hed the ,tmit value. 

AdbOiC irq tc one p e f erred aeptm + d s p'"s?n ra' 
sensing devices in the image sensing opticai systems, and the control step includes the step of adjusting the parallax 
of the principal object in fire images by paraiieiiy displacing the image sensing devices in the Image sensing optical 
systems to separate from centers of the two image sensing optical systems io sense images when the convergence 
angle of s.e.ch of the image sensing optical systems has reached the limit value 

According to one preferred aspect of the present Invention, the paralieltolsplac.emsrit paraiieiiy displaces right and 
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left sensed f mages : and the corrtroi step re. joes 'he step of adjusting the paraliax of the principal abject in the images 
by patalieiy displacing = tgh and - * v - . nag to gene ar rt age for stereoscc pic v ew when the • cm e 
get ce ^r^e of each of he ^ r stnq optica c chad the 'tmr va 

in csder to acrueve - above object, according to she c r — - ' invention, mere is provided an <- sensing method 
comprising: 

tr r ,~ 3 " p~ = ~ ^ and 

the control st^p of setting a limit value of a convergence angle of each of the image sensing optica* systems, 
controlling the image sensing optica! systems by controlling trie convergence angle or a parallel-displacement 
amount of "ha image sensing optica; systems when the convergence angie is not mere than the limit valine, and 
controlling the image sensing optica! systems by controlling the parallel-displacement amount of she image sensing 
optica! systems when the convergence angie nas reaoned tne limb value. 

in order <o achieve the above oDjeci according to the present invention, there ;s provided an image sensing c 
parafus comprising: 

two image s ; > a pair of imag paraifax; and 

-cm o - 5 ! i - - >" -> - - ; e rt m s ir ] o> cmver- 

pence angle of each o? ths image sensing optical systems. 

According to one preferred aspect of ths prs cr nv; on, the ad;ustms is adjust pa s a> oi the 

principal abject in tt ynen ths con vengeance angle of 

of ths image sensing optical systems has reached the limit value. 

--m - - i ' j ' ' ' ), 

ment of tie tms 3 3 rjj 1 1 1 lax of the principai object in 

the images by shortening a base distance between the image sensing optica! systems to sense images when the 
convergence angie of each of the image sensing optical systems has reached the limit value. 

According tc one orsfemed aspect of the present invention, the parallel-displacement indicates parsiiei-dispiace- 
msn? of ;rosge sensing devices in ;ne image censing optica! systems, and the adjustment means adjusts the paraliax 
oi ;he principai object In the Images by pai'alleiiy displacing the image sensing devices in the image sensing optical 
systems 1 hen the converges 

angie of each oi the image sensing optica! sysaems has reached ths limit value. 

Accords to one preferred aspect oi the present invention, the p 1 " I c m^r indicates paraiiaimSisplace- 
ment of tight and fstt sensed Images, and the adjustment means adjusts ths paraliax oi the principal object in the 
images by paraiieliy displacing the right and left sensed images to generate an Image for stereoscopic view when the 
convergence angle of each of the image sensing optica! systems res reached the limb value 

t - > g 3 1" 1 ipoaw v' oh 

are capable o: easy manual centre! of ;mage sensing ophca! systems 

In order to achieve the above object, Becoming to the presem; invention, there is provided an image sensing ap- 
paratus compris leg: 

two image sensing optical systems for sensing 3 pair of images having parallax; 

limit vatee setting means for setting a Smut vaiee of a convergence angie of each of the image sensing optical 
systems, and 

■ o tr means for controlling the image sensmg optical systems by controlling ths? convergence angie or a parallel- 
displacement amoun etthe 1 t singe >= f nan the limit 

sets 11 ^ - e mean j trol! j f i ge ensing 0 1 1 m ms by son ng ne 

parallel-displacement amount of" tne image sensing optical systems wnen the oonverganca ancle has reached the 
limit value 

According to one preferred aspect of the present Invention the centre! means comprises a user interface, 

ft is stBt another object of the present invention io provide a storage medium which stores a computer program 

l m „-v3'a;u5 

Other features and advantages of the present invention will be apparent from the following description taken in 

conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts 

throughout the figures thereof. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig i is an sxp anaiory view of three-dirnensfona* image sensing/image display for stereoscopic view based on 

parallel v-ew according to the conventional technique;: 

Fig. 2 ;s i^fd^e"; . < o id! m=<g« sa^si a 

convergence angle control; 

F "< 0 v % n i r>- *r + -^ir / v s 

base distance control; 

Fig. 4 is an exnianatcry view of h - I image sensing/image display lor stereoscopic v by Ul v 

dispia iq CCDs a m ? , arising apparatus 

■>v f - c r i 

Fig. 5 is a perspective view ol she binocular camera according to the first smboxhmerg of the present invention, 
Fig 7A Is a block diagram showing the; a< range-nan! o; tine sinocuiar camera according to me first empodiment oi 
the present invention; 

Fig. 73 is a view -or axpiarmng the c of using a FIFO according so the first embodiment o; the present 

invention. 

Figs. SA and 83 respectively show right and left parallax image signals; 

Fig. 8C shows a three-dimensional image signal convened t the right anc * parallax signals; 
Fig, 9 is an explanatory vow of control signals, 

Fig. tOA its an t j t. view when a lenticular Ions i Is used as a t ^ - r- mags adapter; 

Fig. 1 08 -c r J of tm 

Fig 11 is a front view showing the outer appearance of ma binocular camera according to the first embodiment; 

Figs, t 2 A and 12B arc- respectively a front view and a rear view of a binocular camera according to the second 

embodiment of the present invention. 

Fig IS is a perspective view or the binocular carneia according :o the second embodiment; 

>~'i *r i " ^ ' m ' r " i t " r = a ,ci, f ' <. 

Fig I5:sa biocK diagram showing the arrangement of a binocular camera according to the bum embodiment of 
ths present invention; 

Fig t6 isa perspective view showing the outerappearar.es of the biricxsuiarcamera according to the third emboti- 
Fig, 17 is a perspective v-aw showing the arrangement of a display unit 115; 

Rg. 18 is a view for e:<oi3;nlng the state wherein- twe o x mage signals are synthesized and 

•> stgnai; 

Fig, 19 is a flo hart ng the image s 3 t 1 3 b n timer - i - - ih ! t .d embodiment; 

Fig 2QA is a view -mowing the synthesizing process of two-dimensional image signals; 

1 Sw ew si- ng she 5* ng process c vo-dimensionai image c els a cord g to t - Garth 

embodiment of ths present invention. 

Fig. 2tA is a front view showing ths cuter appearance of a binocular camera according to the fourth embodiment; 
Fig. 2 IB is a view tor explaining the detection principle of trie overlapping amount according to the fourth embod- 
iment; 

Fig g h - - [ - f " r tl ffl'O TOl 

Fig. 23 is a diagram snowing the arrangement of a system according to the fifth embodiment of the present inden- 
tion; 

Fig. 24 it a perspective view or a camera according to ths * embedment; 
Fig 25 is a block diagram of a circuit according to the tilth embodiment, 

Fig £6 a showing the arrang 1 embo present 

Fig. 27 is a block diagram oi a circuit according to the sixth embodiment: 

Fig, 28 is a diagram showing toe arrangement oi a system according to the seventh embodiment or the present 
Fig. 29 is a block diagram of a circuit according to the seventh embodiment; 

r hewing the arrange-me i acc to 3 hodmen? of the p 

Fig 31 is a block diagram of a circuit according to the eighth embodiment; 

Fig 32 is a diagram showing the arrangement of a system according to the ninth emoooiment of the present 
invention, 

Fig. 33 is a block diagram of a circuit according to the ninth embodiment; 
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Fig, 34 ts a diagram showing the arrangement of a system according to the tOtfi embodiment of the present 

Fig. 35 is a block diagram of a circuit according to the nth embodiment ci the present invention; 
Fig. 36 is a diagram showing the arrangement of a system according to the 1 Sth embodiment of the present 
s invention.; 

Fig. 37 is a block diagram of a em aecordwg io the 12fh embodiment; 

1 ~ I i I ' 0 - ■•].--, C v S = r -sOC -t,-l So U ' ' '3 i' n 

according to the 1 3th embodiment of the present invention; 

Ftg. 35 is a flow chart showing the flow of the processing of the multi-eye image sensing apparatus; 
io Fig. iSC- is an explanatory view of a panoramic synthesis method ;n a through ospiay mods of the melt! -eye image 

sensing apparatus: 

apparatus; 

Fig. 42 is an e:<aiana;ory wew o ; a method of creating an imago for stereoscopic view in a through display mode 
is ota sing appa ) + 1 - s "> 

Fig 43 is an expranatcry wew of a mierhod of creating as image for stereoscopic view in a ; ecordlng mode of the 

multi-aye image sensing apparatus, 
j. 44 is diagram she wi t - rr - n i c nuitl-s i e sensing apparatus 

according 10 trie 15th embodiment of the present invention, 
so P p 45 is a flow "if S - ' " the flow of Ins processing ?or reducing - parallax of a principal object io zero in 

the mas-eve - n " ' 3 r 

- ,i -1 < w of thi ee-d.meos -> 1 uo^tc view basse on 

ct - u fr j - r - 1 " <~ n mt - 3 t| mf?'us; 

Fsg. 4? is an explanatory view of three-dimensional image sensing/image clsp ay for stereoscopic view based on 
25 convergence angle control up to a convergence limit and base distance control m the multi-eye image sensing 

appa.ra.tus; 

Fig. 46 is a block diagram showing me arrangement of a system having a muni-eye image sensing apparatus 
according to the 15th embodiment of the present invention; 

Fig. 49 Is a flow chart showing the fiow of the processing tor reducing the parallax of a principal object to zero in 
30 the multi -eye image sensing apparatus; 

Fig. 50 is an explanatory vie* of three-dimensionai image ---it display for stereoscopic view based on 
, 1 ^ cl <~e ' . h 7 p a < G'pscemtininti - 3 

apparatus; 

n or -ge sens nq/imag. -r [ 1 based ot 

os jnve 3ng ontrc jf - and scf>\ 3 muitse smg appa cord 

:ng to the 17th embodiment of the present invention. 

Fig. 52 :a an explanatory view of a finder and coniroi button for a mtolfi-eye image sensing apparatus according to 
tt r mbodlmsnf c present inver 

Fig. 53 is a flow chart snowing tne flow of the processing of a method of moving image sensing optica; systems 
40 ithep + .w a wr r : - ^ f < ' 2 ^ , opaharus 

Fig 54 ;s a flow chart showing the now of the processing of a method of moving image sensing cortical systems 

control :n the {*} dsrectson in the mufti-eye image sensing apparatus. 

r ^ . r - s ^ r _to v r - p-> ^ p- 1 ' : w wwmm rruomm - * < m. 
control m the •-; direction in the multi-eye image sensing apparatus: 

Ft 57 shov/s program code modules stored in a storage medium 1 m a mufti-eys ;maca sensing apparatus 
3 nvention 

Fig 58 shaw ran edsie s m used m s m 

: 3. 5<j " 1 ~ ^ ^ - ^ \ iff - o a k o s 

of the present invention and different from Figs 57 and 58 

DETAILED DESCRIPTION OF THE INVENTION 

At s - ir s -m oem is:: -•• t .c . " 0 -3 mbccSmo o* k p esetit 1 ivuntor 

v r fc d t n„r it nc accompanying drawings In ents. tne press 1 to 

is applied io a birsocuar camera (so-called stereoscopic camera) The embGdirnenls of teindciilar cameras will be 



EP0 830 034 A t 



explained for the same of simplicity, and the present invention can be applied to a multi-eye camera Including three or 
mors camera units. 

< First Embodiment* 

The - r i > '"fci:"Hc k o o > 1 

below with reference to Figs. 5 A, 5B, and 6. Nets that pig. 5A is a front view of the binocular camera according to the 
Ins; embodiment, nig. SB is a rear •/jew of the camera, and Fig 6 is a perspective view of the camera. 

As shown in Figs. 5A and SB. the binocular camera c! this embodiment comprises a camera main body f, two 
camera heads 2a and 2b which are attached to the right and left sides ol lire camera -1310 body ■ and have two lenses 

f r t 3 r n a v"\_ N t 

that the camera heads 2a and 2b nuilrian image censing means, and the stereoscopic display 4 builds a display means. 

The camera heads 2a and 2d are attached to the rightmost and leftmost portions of the camera main body 1 ;o 
L a r u j - k 

scopically displays images obtained from the right and left lenses 3a and 3b. The stereoscopic display 4 may adopt 

>' b it - f - r l 1 I - e ^ u l - csl 

as spectacles with shutters and tne like. 

As shown in Fig. 8. the camera needs 2a and 2b can independently pivot about the horizontal axis agreeing with 
the *idtnw!$8 direction of the main body Upon Image sensing, the operator directs the tenses 3a and 3b toward an 
ret 1 < J i - - r 0 t r v 1 1 1 ii v - v. o 

display 4 t g t nla observing re-- 

with the j - r> 'Sec - - - 4 h v 3ted in correspond / a rno\ m ent 0 positio ^' f m 

operator. 

A signal processing system o; tie binocular camera according to the first embodiment wrii be described below with 
reference ?c Fig. 7A. 

The binocular camera comprises lenses 31a snd 31 b, two image sensing devices 32a and 32b, two A'D converters 
33a and 33b, two FIFOs 34a and 34b, a timing generator 35, a slgnsi converter 36, a CPU 37 with an internal memory, 
a display t~i 38, a stereoscopic display 29, a recording controller 40. a recording medium 41, and a camera 
le 1 t _ -, s annm-dimc -ion f 

image signal syntheeiz rg Tisane and - ns, the stereose^ 

image signal display means, and the camera controller 42 constitutes a second selection means. 

The image sensing devices 32a and 32b, the A/D converters 33a and 33b. and FiFOs 34a and 34pare respectively 
onnected ^.wii'v di ' 3 1 vcrer 

36, Thestsn 5 connects e nverter 36 b-renonct^p 

medium 41 is connected to the signal converter 36 via the recording controller 4C The A/0 converters 33a and 33b 
are directly connected :o ins CPU 37, and the FiFOs 34a and 34b are connected to the CPU 3? via me timing generator 
35. T ■> - r " 3 -> t r t v •>--->■> a 

c e 1 - ^ 4. s r r i by tne user to swlich the 

mode between a panoramic oxide and a stereoscopic mode, 

d ec-vsrt images sensed via the lenses 
31s and 3ib Into electrics! signals by a photoelectric effect. The A/'D conveners 33a and 33b convert these electrical 
signals into digital s goals. These dig-ted signals are two-dimensional image signals The FIFOs 34a and 34b temporary 

tuning generator 35 supplies a write control signal to toe FIFOs 34a and 340 Tire CPU 37 controls the A/'D converters 
33a and 33a, FmOs 34a aoo 34b. timing generator 33, signal convener 36. display controller 38 recording controller 
40 and camem controller 42 The signal converter 36 converts iite two-dimensional image signals written In the FIFOs 
>a ru -1 1 e-Oimensional smag gnal, and the disc t - n « si Irmug 

goal generated converter 33 or r p ay 39. Tl- record gc 1 1 40 writes t a three 

rr h -q ~ 1 r 3- i , rsni t san 



s n - ddua units InThis embodiment asan exar 5 on sncpsr 

amlc image sensing mode and a starsoscoplc Image sensing mode is prepared 
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<Tr.'se-cimens(onaHmage Pfescassin3>.. in First Embodiment 

A case wifi be explained below wherein the stereoscopic image sensing m«fe is seiected. 
Whs^ the stcrecscop ; image sensing mo s g r * k t P formed 

5 o tV nags 9 s t 

oeisd f . H J set j t?r _ i s by the A/D converters 33a 

ar:c 33b T.nese- digital signals are two-dimensional image signals. At tins time, the right and left two-dimensional image 

- of of - _ ! =) a o^ i s r r d ; c s 

ehroncaisly driven under the control or the CPU 37. These two-dimensional image signals are temporarily written m 
>Q the FIFOs 34a and 34b torn generating a three-dlmensiona! image signal (to be describee later;. As shown in Fig, 7B, 
t c <- t- < i r , , a) ^« _ ^4. < qnc ^ for 

s < rj l r ^ t ^ V f 1 

A -i < n t - i ' It Ji - -i< ^C^.Cs.»f ev.fitfiin 

one-trams areas (34a- 1 and 34b-i) of she FIFOs 34a and 34b, two-dimensicnaf image signals corresponding to an 
f5 rags for the nex frsrr 8 ate wri sn in tf emaif ig no ame r '4a 2 and 341 of the f 3- J4a s d 34b 
When the two -dimensional image signals corresponding to .mages tor >wo frames are written in tno FIFOs 34a and 
34b. twc-dimgnsiotisi image signals corresponding to an image for the next frame are overwritten on she areas (34a- 
i and 34b- 1 ) of the FIFOs 34a and 34b The writs control signai of the FIFOs ;s suppled from the tinning controller 33 
s C- J 3" 

so t i ^ signals written; in the FIFOs 34a and 34 se-dimsns ^c 

signai by tog signal converter 35. Upon conversion ci two-dimensional image signals ;nto a three-dimensional image 
signal, as shown in ~ £A to FC. a right parallax Image signal (Fig. SA) and a left parallax image Signai my 33} as 
the two-dimensional imaua signals written in the -OC 34a a n umts c? vertical lines 

as anew r c ig en t~ t -, m - jp nage signa 

ss v< a * -stg -I - tn « ' r 1 n* --m -a- , j J * ua a - i ^ss>r^ - _t 

8A and S3) into Image data shewn in Fig. SC. the three-dimensional image signal (Fig. SCi Pas a width twice that of 
the original right or left parallax image signal (Fig. 3A or 88). in view of this prcc < i *» > ' -"3 

to half .In the horizontal direction during the process of generating of a three-dimensional knags signal, or a three- 
dim? ^ i 'T( m oe generated using a portion of the right or left paraiiax Image signal. In the example 

30 (Jro* " ^ - i I r " "3" " - c u ~ is gene c „ m 111 

p hon- (so rounded t), tmksn I r 3 ' ght and left pa nals, the 

of the image ssgna 5 sjha an nchanged 

The signgt converter 33 reads out two-dimensional image signals corresponding to an image for one frame of from 
the : areas 3445-1 and 34b-1 of ths FIFOs 34a and 345s under tne control of ths CPU 3? after these two-dimensional 
3SI image signals are written m the FIFOs 34a and 34b, Parallel to this read, two-dimensional image signals corresponding 
m i- 0 'he ^ * -one 2 c v he- in the areas 34a-2 and 3ab-2 of the FIFOs 34a and 3*t>. For this reason, 

the timing of a write control signal WR and the timing of a read control signal RD transmitted from the CPU 37 are 
3 b " rsoccopc 

aisciay 39. This stale is shown m the explanatory view of control signals :n Fig. 9. In order to prevent trams fossas 
upon image sensmq ; the signai converter 36 must alternately man 00; right and left parallax images lor 0.5 frames 
_ i n a -.i ^ > r Or ifmro 

Into the FIFOs. For this purpose, the read control signs* RD must be simultaneously saagiled to the FiFOa 34a and 
34b, and its frequency must be set to be twice the write speed tine frequency of the control signal VVPg to the FIFOs. 
Under such timing control of tne I signals WR and RD, the signal converter 38 generates a three-dimensional 
45 image si ith a si - s 1 - + 0' e ! e c -e c , e g 1 < goals. 

The three-dimensionai imago Signal generated by the signal converter 36 is suppiled to and recoroed by the re- 
cording 0 It <tg via tne display controller 38 under the control of the CPU 37. Tde display controller 33 displays 
the generated three-dimensional image signal on the stereoscopic display 39. 

ia KH 's a top vew c m^ v t 1 unj 

so ' -t: v t r ^ r <- c 

pi t e v t " rfa e p s »r- s c « e 

severe! types of adapters are available Fig, sOA snows ths case wherein a lenticular lens 61 Is used as ths three- 
dimensional image adapter, and Fig. 108 shows ths ease wherein a parallax Gamer 62 is used as tne three-dimensional 
s m m Ci \= entcu'ar lens 81 anm 
ss - Lm - t ^ - e c ^e " *- o< - ci display 39 and thee- ? . ip:^' ' * . ' 

Sines; the oisplay unit of the first embodiment also assumes displaying a two-dimensional imago signal, tin is aoaptar 
is detacnecie in consideration of the compatiblfity the iwo-dimensonal ina^ signal. The operator must detach 
tne adapter wTen he or she sets a switch 43 in the panoramic image sensing mode that does not require any stereo- 
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ecopic vsew. 

"it rccc «!pQ of s 4T>Ait_ mt> f " c - r ~ u * ^ " ^ 1 "ee* h s 

c % - -t. ' he n oi-m a- ha ci ng r diem 4 a magne " aps „ i d s< ssm 

ic f djC Jf f? st c r v ho s may t> used " s c -O 1 1 i i ig J * K- 

5 digits Dnrsat :n an empty area c s recoi g 1 a* a He 

Ths recording on the recording medium 41 starts or stops when the operator inputs a desired operation to the 
camera controller 42- Ivors that the operator can perform only obseivatlon of the three-dimensional image signal on 
the stereoscopic display 39 without any recording, 

->->--•- j } , n ^ } \\-n ^ - t r -v - e - r- i> 

is signal recorded >n ths recording medium 41 wii! de explained below. 

in orde; to reproduce a three-dinwislonai image recorded in the medium 4i : the user must Know the presence oi 
a recorded image rise. Sine r j 4t s lily of ft e n gna 

re re ed in these f he to m I r - - i 3 in -he re iin o 41 and Ti isle s 1 
i ] r t d j i i i irdr 

is nr j - t ~ n ^ i i T V.F 1 " f ^ t 

ht c S t the st play 32 "'"o ope o r ss *ci=> a f Is t > oe reproduced from 

the displayed list of file names, and sends it to the camera controller dp. The camera controller 42 transmits the selected 
ftle name to this CPU 37. The CPU 37 reads out the selected file from tire recording medium 41 via the recordsrsg 
centre-tier AO, and displays =t on the stereoscopic display 39 as a three-dlmees tonal Image signal via the display con- 
so troiter 38. In mis manner, the sensed three-dimensional image signal can be easily reproduced without rsquifingany 
dedicated equipment. 

in order to record ano reproduce a voice together with an image, microphones 5s and t are set on the front 
surfaces et m camera heads 2a and 2b, as shown In rig. 11. 

SS < Two-dimensional image Frccess;ng> .in First Embodiment 

The processing In ths binocuiar camera until generation of a two-di n age signal and the prcce^ 

' > - j v interna image sgna >. b<= .v. o* a below. 

In this case, a two -dimensional image signal Is recorded or displayed by processing -t irr - - "dl from .-one .of 
a? She lenses 3*a and 31 b. The cperatof can select the lens to be used by making an input lo the camera controller 42 
The difference between ths processing In the binocular camera until generation of a two-dimensional v \% • gn 
and that oi a three-dimensional image signal is that the signal converter 36 operates as a elgnai selector For this 
tjxr tv mensionallmage sig the FIFOs 34a and 34it ppiied _ p ont roller 38 or 

recording controller 40 without being converted into a three-dimensksnaf image signal. Other processes are the same 
es as those m the above description. 

Wher he operator swiiche n is a put to the camera control 4 3 iu n 

recording of a two-dimensional image signal, the CPU 37 switches ths tenses 51 a and 31b in synchronism with fetching 
o t t 3 " n t r - r-->« su t-d open 

switching of the lenses 3ia and 31 P. 
40 rns jfocss <g -se binocular camera up i repreduc one s .voOimensic-na mage signal is the same as that 

of a three-dimensional image signal In this case, me Its; of file names of two-dimensional image signals Is displayed 
tor the operator, and the selected file is displayed on the stereoscopic display 33 

Advantages of First Embodiment 

45 

As described above, according to the first embodiment, :ho user can stereosccplcatty observe a three-dimensional 

' t >mhrumhu i -time. 
Since the user can watch the stereoscopic disp.ay 32 web bote ayes, the degree oi fmecom m posture upon image 
sensing is high, and the user can confirm stereoscopic expression even whan he or she moves white holding the 
so binoouiai camera during Image sensing. 

-iju tmc *~ s " " r r t 

right and left f r i i * vvn in Figs. 3A and S3, but can also be attained by a simp oper don eg., 

by changing the distance ostweer; the ooisct artd the binocular camera upon image sensing or iocm.mg of tne lenses 
3i a and 31 b. Also, tire camera can be easily switched to a conventional single-eye image sensing mode 
ss \ r i it- ntion can i both moving 

images and situ Images. 
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oSeeond Embocimsn;> c j , t spectacles with Sautters 

r n -."•,.,'r-' .'j'TOi.y :i-e^uyj'>c*r e rf.ri rr'-^rrn t -s^o-vm be 

descnbad betow with reference to Figs. i2Ato 13: 

Nets thai J2A s 3 front sw o c c sr camera accord 5! an 3 ! n 0? the seco 2 - jimsr 
Fig. "25 is a rea- view ~ ths camera, and Fig -3 is a oemce-cove o \ of e camera. 

As shown in Figs. VZA and -2S a . fcodtment com t air; body 7C s 

single c nershsad? vh attached to th a ut r -f bod\ ? 5s and 

75b on ths front side a connector 72 * i r ; ar 5 ed ie r ds of th« tamers bod " tc cnneef 
spectacles 74 with shutters to the camera main body 70. and a stereoscopic display 73 arranged on ths rear side of 
trse camera main body 70. 

-^^O 7 can o- aom .: ,;• ;a 6\ :.. r \r Oi S pecf = c 

r <m i r cc; I * m i m ifn": s - spr : -> >~ tf .100 JV ; n 1 't; f -m an t t~ 

- - oenoemiy display a t < imag its on the right and left eyes 

1 -en t or ^ M 1 r I c a a t - r 

becomes twice that upon displaying a two-dimensi mags signa 

The arrangement of the binocular camera according to the second embodiment ml do described below with rot- 
ersnee to Fig. 14, Fig. -4 Is 3 block diagram of the binocular camera according to she second embodiment 

Tit o-cc a', "it »t t. - T f r p — i c -i - r 

92b, two A converters 93a 3r; o 930, a 3W ; i0 h. 94. a g S; g ^ 96. a display controls 97, a stereoscope display 
38 a 'Sdc o p medium 100 a can„-a c 

and 100b. stop controllers 104a and 10-th. focus eontroiisis 1C5a and t05b. ansa CPU IDS with an internal memory 

~» v 32d are connected 1 t ^ t i 

switch S4. The umt S3 is connected to me FIFO 96 and ths recording controller 95. The FIFO 96, sne dispiav controller 
9 id ' ^ m e-m v „ o j r o - i - t d t - s -o n^m o 

to he jordiog medio 3 in. the d 1 0 1 4- h d nc ec I -j- e can a controlie 

101 , the shatter controller 102. the step controllers 104s and 104b, and the foe j- Or 1 . r , — 
nectad to the CPU 106. 

T X D f " s d th - 

1-> i ^> b Et 3=oj \ U O! « ct Th^AO r 

electrical signals into o signals. Ths jstaS signals are two-dimensional image signals. Ths switch S4 is used for 
alternately transmitting right and taft 'a m-m m image signals - from > e right and left A/D converters 93a 
and 93b to the on.; go The unit 95 performs calculations for AF (auto-focusing) and AF (auto-exposure) and white- 
^ - i ~ ~~ -mp - i - 1 - i r - < ) o - .it 

scopic display 98.. and the display controller 97 displays the two-c me < ' i r m 1 1 " " 1 
the stereoscopic display 95 Ths recording .tin 99 writes a two-dimensional image signal - me recording medium 
100. Ths camera controller 101 transmits an snpu! signal indicating contents input by the operator to the CPU 37, and 
theshuth?t u vdmlnr- n ~ % "d „i - , 1 ° , - - , ^ 

ths stops 103a and tCSb. and the focus confronts 105a and <05b control the positions of the lenses 91a and Sib. 
1 - - , i m,ajo p i oi 'fd — v c i^r o mon-,01 

tfoller 102. stop controllers :04a and t04b, and tocos controllers '05a and lOSb. 

Thj pe.oator selects a thi dimen - -age mode or a norma! image mods via a mode selection switch 107, 
,r - " 1 r t r i ^ e bma; -dimens o-ai idle 1 xle tn 

T r s ^ Otu A' , lop 9 

sndwidua? ' - hnages sensed via the lenses Si a at 9f a by the opei'ator are formed on ths mage sensing devices 
32s a or rtherm ti^-j^c^of''' = ^ O t neatly ^.r 

verted into electrical signals, which are then convened Into digital signals by the AO- convenors 99a arid S3o These 
digital signals are two-dimensional image signal . Smce the switch 94 aheroately and ssoaraielv sends the heht and 
^ " 1 f e -i J a r. m- " n 

image srgnat Oecornes v ^ - 1 upon displaying a two-dimensional image signal Hence, the read sased oi electncai 
signals 'mm f n-temnq O, „ Odic- O, _ j- ti 3 J? i 1<j ;o - 00- ^ n — 

01 a rn-i , -1 ^1 j u 

sq 1 » e-dd smr v e - r rn-t — - a - - a Jo - d *m - 

sgm s,^Jait''ir -«d.8 ^~ r sm g^r j- , ^ s - . , r 

Theriglrttanaisfttwc nafimage naisarssaratc "1 oesablect J <. u do. i ens 'o-^ auto 

o-t,sng a rd AE ta.m <,< c- a» a- -oa a -eao.sr • "• a m. ^ "o- , done us 3 

lurrt nance values of pixel signals in a given area in the image, and the calculation results are sent tc ths CPU 106 
Ths CPU 106 determines an appropriate control amount on ths hasis of the calculation results from the unit 35, 
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the currently held states of the stops 103a and 103b, and the positions of tns tenses 9la and 91b, and sends She 
control amount to the stop controllers 104a and 104b and the focus controllers 1.8Sa and 105». The step contrciiats 
104a and 104b adjust the stops 103a and 1.03b on the bas;s of the control amount, and the focus controllers 105a and 
105b adjust the positions of the lenses 9ia and Sid on the basis of the control amount. 

Since the unit 95 alternately receives the right and left two-dimensional image signals via the- switch 94, the control 
processes in the step controller; 104a ana 104b end the focus controllers '05a and 105o are alternately dene ;n 
cooospondence with the reception timings. 

Tne two-dimensional image signals subjected to white fcaiar.es adjustment by tne unit 95 are displayed on the 
stereoscopic display 98 or are recorded in the recording medium 100. 

When the three-dimensional image display mode is selected at the camera centralist t01. the right and let* twe- 
i r or . - Image signals arc sequentially sent to the display controller 9? via the FIFO 96, and are alternately dis- 
r a i ^ t ]? „ r , ^ or- >b"A^iri ml 

l a; Ion signal to tt n g t u signa 'or 

driving tne Spectacles: 74 with snctters The gener c i - \ butters vi t rr a 

r- I "~ c 74 tit s , - c - d _ p.. 

r vn r i t i j _ n j ir t o [ 0[ _ j > ^ iL,mrc- 

h ' i = _m V n " a - ' - - — ~ - ^ p te * - c s -it: 

- r t i/ r t s t f d ; o n i 

i>! the FIFO 96 td elspiay tea right parallax cu signal alone on the stereoscopic clsplay 98 " CPU 106 
end j ^ r t - ^ r r I _C > n me F- FO 

96 Is a right parallax image signal, the CPU '06 s is a write ol slgr th FO 96 wh h seords he ght 
-v i < Image signal, and thereafter, sequentially outputs the recorded r parallax image signal to the disetay con- 
treiier 97. The right parallax image signal output horn the FIFO SS id displayed on the stereoscopic display 9S n the 
display controller 97. On the other hand, when the two-dimensional image signal sent from the unit 35 to the FIFO 96 

t t a ^ t F in s j « ^ \h O t H 

not nxc m« <. t c o ,< tpc y Is j* * s th' 3 I i c ^un- - r -~ed'u T e \ h mhto m -k 

96 again via the display controller 97, 

Whan the display mods of a left parallax image signal ;s selected at the camera controller 101. the CPU 106 

ov 35 as n "-s case i A"<re " 

the display mode of the right parallax image signal is selected.. 

m ths manner by centre' ng ne F' = 0 - - * < ' < - ,r -om- 

image display mode and the two-dimensional Image display mode Co the other hand, the operator can temporarily 
display a three-eimeneonai ;mags signs on the stereoscopic display 99. and ihematter, can switch the display mode 
tc "is" v v tn o -1 - e c s o >"m - f - - H - e t - m m u - "> - -oi 

expression using She three-dimensional Image signal, and thereafter, can adjust framing using the two-dimensional 
image signai. 

Note that the display period upon displaying a two-dimensional image signal is substantially hair that upon dis- 
playing a three-dimensional image signal. 

The three-dimensional Image signal j 5 reproduced in the same manner as In the ft embodiment. On the other 
turn , <-i o *e rr ) r -< -< i m f „ > 13 0 ed > t, oe displayed and recorded, the 
switch 94 Is switched to display ano record a two-dimensional image signaf from only one lens, in this case, tne CPU 
111 1 ^u notv -Cat t t » ^ j^e^omJ ^aq° u qoO t cm i i f 

att sicca al \ f " » CPU 106 does jndanydnvir a he utte contr er102 

to stop driving of the spectacles 74 with shutters 

^Advantages of Second Emoodimsnfu 

As described above, according to the second embodiment, a three- or two-dimensiona! Image signal can be die- 
stayed o aeorded 

E, ( >n /h a r \ it m- - - c < ~ > -u 

1 a n F / 1! or 0 1. s 

the spscfacies 74 with shutters during displaying on the stereoscopic display 98, he or she can observe a flicker-free 
two-dimensional mage signai 

Note that the camera according to the second embodiment can be used in image sensing of com moving images 
and still images. 
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<Thifd Embodiments. . Automatic Mode Determination 

Th© bio 1 r cameras in the first and s c ■ are set in one of ste k 

y <= '» ~c - 1 - * to i -' d ~ ci 

insert, an optima! one of the two modes is automatically set on t basts of the deviation between the pivot angles ;dit 
credos;; oi the ugh!, and left lens systems 

t i h t m i . i f <-r* <~ vwfl b( s> 

pained beiow with the aid of tog. IS 

T-ifcbi-x f t ^ - c i:n:omp s^: re & sf, -e "to a a _ o rrxwisr 

c me nr body 120 The camera h md mi b, and image sensing device 

1 1 2a and 1 1 2b. and the bsnoeuter camera main body 1 20 compnses a esgnal synthesizer n 4 for synthesizing signals 
from She image sensing devices 11 2a and ti2o, angie detectors ti5a and ti5b 'or detecting the pivot angles of the 
ptay o:! i - a 1 i m tx jsss button n 

n j t A ar - f t h - f t i n-- e ^mv^ mle 

1:3 are connected to ins signal synthesiser 114, The camera heads si 3a and 11 3d are respectively connected to the 
angle defectors H5a and 11 bo. and the angle selectors it 5a and 115b and the release button 115 are connected to 
the system controller 118 

Th* 1 a r^F no awi^edevuffiJTO 

adjustment units and stop adjustment units 'neither are shown;. The image sensing devices 11 2a and 112b convert 

The signal syntbescer 114 generates a two- cr three-dim ensionai image signs! on the basss of the image Signals 
obtained by the image sensing devices ' '2a and 112b The angle detectors 115a and 115b respectively detest the 
pivo? angles o; the camera heads i 1 3a and 1 1 3b, in this embodiment, rotary encoders are used as the angie detectors 
115a and it 5b. 

The display una 1-6 displays a two- or three-dimensional image signal obtained by the Signal synthesizer 114. 
Fig. 17 shows the arrangement of too display unt 116. Referring to Pig. 17. a liquid crystal panel 131 has a large 
number Of display pixeis, and a lenticular lens 132 with U-shaped lens umts is arranged on the treat surface Side of 
the liquid ' .crystal panel 131 . A backlight 133 is arranged on the rear surface side of the liquid crystal panel 131. Sines 
the display unit 116 has seen arrangement, it displays, on the liguid crystal pane: 13;. a -hree-dimensionaf image signal 
■43 synthesized cy alternate iy arranging two-dimensional image signals 1 and 142 having oitpo parallax 
taproots I 1- t t r r The 

above-mentioned pitch corresponds to the pitch of the lenticular lens 132. The three-dimensional image signal 143 
dispte 5 the pan a the observer to expensr t copic / It tar lens 132 

V J t I r i 0 o - se Abo ere 

skilled sn the art, as described sn Japanese Paten; Lard-Open Mo. 3-65943, and the like, and a detailed description 
! i A itn s a parallax earner sn place of tha lenticular 

lens 132 mr u J , I .tot i. i - a - g- < l^rr er , p ,p 5 = : 

in c \ =. 't r ^ >r t " r ^ Vt- " j. -\ \ > rap- 

tor the left eye is observed by only the ieit eye, and the image signal for the right eye by only the right eye. are available. 
However, since these systems are already known to those who are skilled ;n The art. a detailed description thereof wlii 
be omitted. 

rumackt g. 15, the ding un 117 records an image c_ e e e, t - thesizer 114, 

i Bcontrois sblno mera ot this embodiment. The re a s s button 1 1 9 3 « s 

a recording start s;gnal of an image signal when it is operated by the operator 

t t * d m- 03 M l > or v rlf be 

explained oeiow with reference to Fig. 19. Nose that ail the operations of the binocular camera according the third 
embodiment are controlled cy the system controller 11 3 unless otherwise specified. 

When a power switch foot shown) of the binocular camera is turned on (step St 02). focus and stop adjustments 
are performed for an object (step S1C1 ). 

The ancle detectors 115a and Pi 50 respectively detect Che poo; angles of the camera heads 115a and 11 3b (step 
STQ2), 

In step Si 35 it is checked if a relative deviation amount 5 {- % - e b ) between the pivot angles {% and %) of the 
1 s e c- 8xar< ois 
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fSkfe: 

if it is determined in step Si 03 mat the deviation amount 8 falls within, the predetermined range, -m-- n i 

o c?Ja if -as s -afs -i -r r b a I ^ 

sig^U/n;ht.3i2e '1 i -'0 a u" r 3 s ^ m mn -0 m >«r "ruhiin ct" a* r-aa ira 

^ - r m 2 t o _ - - 1 f ~ r r I 

sc - ptav if ' ee 3 cr can ste'soseec . < set object im 

When tne deviation amount 8 fails within the predetermined range, this means that the ttit angles o? the right and 
w left optical systems have no d e veen is easy to obtain an i mag sing state for stars 

lis - i <~ T he ct at to has jr \ " , -*<s J Jtc $ t is n<-d n s rp S Of h t 

the operator h <^ ed on the re e o tor 9 - jtmensiona; image s g I43$vnl zed b} he signs 
syntnes « 4 is recorded in the re ng en i o ° in T s 3 to * Jcked in step S108 if the image 
sansingc r n 1 t z - l f e age - r g t r tne power t 

?s a i ise. the flow returns i a.d S't'OL 

On the other hand, ti it is determined In step St 03 that the deviation amount b fails. outside the predetermined 
< 8 jrmic i mnj si.ijj't' arn'o:-" m:M- etm eg 2a a t a e i 

b ? _j AC' 1 h -ji u m0 - - »- , * L t it - "f ti t 4 {step 

~ v i ; - yr a 0 i ' n 3 5 ne 

as -low advances to step SI 05, 

When the deviation amount 5 talis outside the Dredeiemtined range, in the extreme ease, the tilt angles of the right 
and left lens systems vemcatty iargety deviate from each other; and in such case, an image fusing state is hardiy 
obtained. 

if it is determined m stec 3106 that the operator h $ ~ s-loase buttc S the flew returns tc tap 

ss SI 01. 

In this manner, one image sensing by the binocular camera according to the third embodiment ends. 
As de-scribed above, according to the third embodiment, since the display unit It 6 adopts the lenticular lens t32. 
the ope r need not we c a aft - spectacles such as polarization spectacles or the itks even when a three-dimen- 
sional image signal is displayed. 

30 

<Fourth Embod:m,9m>,. Automatic Mode Determination 

~-tr'd , - t m- >-e of the three- and trwo-dimensionai display modes on tha basis 

of the deviation amount of the tilt angles of the right and left iens systems wh*h are pivotal about a horizontal axis. 
3$ However, tha fourth embodiment automatically determines tire current mode io he set in the binocular camera from 
the stereoscopic: v<ew display mode, a panoramic view mode, and a norma* image sensing mode or; the oasis or the 
deviation amount of the pan angles of the right and left iens systems which are pivotal about vertical axes. 

The outer appearance of the binocular camera according to the fourth embodiment of the present Invention wis 
be explained below with reference to F;g. 21 A 
.«; Reiarni tc Fig tA he difference in t c r camera his embodiment from ecu - appearance of the 

binocular camera of the third emnodimeni shown in Fig. 16 is thai the camera heads 11 3a and at 3b are respectively 
arranged on the upper portion of ;ne binocular camera main body 120 in olace of arranging them on the right and left 
v % head 112 n 1 3b can pivot ab s 

vertical axes. Note that the arrangement of the binocular camera is the same as that shown in Fig. IS 
js t \. a t t - "> j f r v " 1 : - n -n < t > 

described below with reference to Fig. £2, Note that all the operations of the binocular camera according the fourth 
- 0 d -i ' 0 e r 

When a power switch i shown) ct the binocular camera is iumed on (step 33001, locus and stop adjustments 
are performsd tor an olgeci (step S20n 
53 a 1 p t i of the 

camera heaos ft 3a. and if 3c {step 3202'; 

St ? c S ~ " 3 d e o vm * a v - a i c <Jn 

the basis of tne pivot angles % and % of the camera heads ■ 1 3a and 1 f 3b, the focal lengths (f: common to the two 

than a predetermined vaiue 

If :t s determined in step 3203 tnat the overlapping amount is larger than the predetermined vacua I? is determined 
ma* t ie rj rechons of tr e camera heads V 3a a ; r J:3 s^ 'ah e dm sto-eosoootc wew -a an 'mage J .s "g s'ate 
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is easy to obtain, and two-dimensional image signals 14? and U2 obtained tsy -he image sensing devices si 2a and 
1 12b are synthes zed by the- signal synthesizer iu into a single three-dimensional image signs; 143 in which , Sght and 
i- -s aio <■ _ -s tor r I - •• - ^ to r <*\ on 

sszeotf at dimens e. "mag « C s r is it oo „ vs 

* a wde panoramic image or images in two different directions (step S205) 

!i is checked in stop S2C6 if the operator has turned on -he release button 119 If it is determined in step S206 that 
te operate is sdc the release butt !he three-dimensional image signal* 4 < i „r 

to" ^ 1- ' c- 1 t r - d - i '1 ' ^toe S207) 

The aafie it s checked n s + ec 32C~ to ■> a j e •> < v - r - v s md 

10 if fhs image sensing is to end, the power switch of the Binocular camera is fumed of'; otherwise, the flaw returns to 
step S201 

On the other nana, ft it is determined in step S203 that She overlapping amount is smaller than the predetermined 
vnli ■> \ ■ \ eW s e-- ~ o=„ ^ c '-mc- tr. I k m 

If it is detorminad >n step S209 that toe overlapping amount is not zero. s:nce the two fields of view of the camera 
ts p ac - i 1 " -t - r * h „ - t ^ to -< hi 

113b are not suitable for stereo < u mora % % 

1 s procss - goals, twoto to jnafe 164 and 13$ obtained 

i sin tfc'-h^qi t vo-dimen slcna - 116 

he sig synthes ste and thereafter, tr 3 5 to s 

so If :t s tton, on • step S2 9 that t ; verl.e.pp:ng amount is zero, to, dimensional image signals 1 61 and 152 

■to < t i r - - v a t pt a 1 

P tip d f la - to 

totf r r - — r "a _ _ o = 

if it is determtned In step S20S that the operator has no; turned en the release button US. the flow returns to step 
2S $201. 

in this manner one image sensing by the binocular camera according to the third embodiment ends. 
< Advantages of Fou;th e rod potmen to 

■ ;;t ' : ' As deseeded al a oeording the k th < odimeto toe camera sad n - c J to re p ? act 

vertical axes, and whether the directions of the camera heads are suitable for stereoscopic view or panoramic view 
can be automatically determined on the basis of the pivot angles oi the camera, heads. 

!h both the third and fourth embodiments, image sensing of stilt images has been discussed, but the present 
nvanto f c i <~ r - ire i * ^> - r ■* < ov *hf= 

ss signal synthesizer 114 is recorded, but image signals defers synthesis may be recorded. 

<tofth Embodiment 

sf i to 1 2th embodiments to the present invention w-M be explained hereinafter: Tnese tide to 12th embodiments 
40 o intpfovsments upon applying the mutti-eye in h _ x to an 

information processing apparatus such as a personal computer or the like. Note that the fifth to 12th embodiments use 
a binocular camera ss an example oi the multi-eye mtags sensing apparatus. 

1 ! shows e arrangement a system ceo " n to- h 3 embc t < c m nera of the present 
invention. 

"'S Reie;r;ng to l^ig, 23, mfercnoe numeral 1001 denotes a camera main body oi the onoctoar camera, and 1002 and 

1003, right and left image sensing optica! systems, each including a lens barrel that houses a tens, and an image 
sensing a a CO o e c t n notes a liquid I d can perform 

thrsa-dimeno-ona: display: 1005 a storage medium, 1006. au interlace for exchanging a digital signal vtoh an sxterna! 
I d t - „ - -> r r m f ) 

S(J and <, I T ii - - - ; ■-„ ^ " " r„ - i a jr 

synci z j - nreceiv gnal 1 100; and 1000 

liquid eg ^ ,e tne sync^ron>zatop signa! t 01. 

As shown so Re, 23, the binocular camera is constituted toy toe camera main body 1001, and toe toe: ; ma g. 3 sensing 

5a jn 1= - !U - ^ S- Z 1 r " 2 r. " " t 

the right and left sees o? the camera main body 1001 to sssafe a kSrtg bass distance. 

Also, the liquid crystal display 1 : 0Q4 having a display mode that sfiows the operator !o stersossopically oOss:ve 
right and left para!ia>; images obtained from the image sensing optica: systems 1002 and 1003 Is arranged on the 
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camera main body 100C Although many stereoscopic dispiay schemes have been proposed this embodiment adopts 
a >v-«'-s - :: - - .. - -m - re~" re ss CC vv l f ft => 1 ^ ■< ••• c TT4 

i , i, «t "L/C -ers'sw^sra i r r d rr-tja 

[ T c r 2n h I )t t do t 1 — t i^a^ 

s inc i - stems 1 0C2 and 1 on th t;utd crystal display 100a. The r. real reha f jtween 

tr"a 3 S s5-sr- re- svsfs -re XI mo *00u a d t> a 1 r - m i * 'i' 1 Miirc^ ci 

in correspondence with the observer as shown in Fig. 24 The relative positional relationship between the two image 
sensing optical systems 1002 and 1G03 is fixed, and remains the same even when tne liquid crystal display 1004 
rotates in tne tiit direction 

13 Du g image - re reduction atte mag* sensing be obse r m ebsen a thres-<l;me s n nage 

- y I i 4 -hat - - < j i :? r i t 3 ag c re \ a so ay 10 r A i 3 

observer obsen/as an image on tire- TV display "009, the camera mam body 1001 outputs "he image signal 1100 to 

4 i J term - t h r it 01 to the liquid c 

shutter sp - > 0 are also used when an image is observed or the 

m liquid eresrel drepiav 1004 

Tne flow of Signals ana the now ot processing in the camera upon sensing a thiesdimereaoasi image will be 
described bo low with reference to 25 

Referring to Fig. 25, reference numerals 1 020 and 1200 denote CCDs: 102a and 1204, CCD drivers; 1021 and 
c ;0S/AC ifewts tJ22and12 mpine circuits; 1023and '203, AC' converters: 1025, a toning generator; 
so < i ! f> c ", ) r m it n ,d o a c 'o i 

l s 1 1 a q J x ^ i >- - _ -Ore ^2 

and 2003; process memories. Reference camera: £004 denotes a ~n - r r which executes, e. 
g, JPEG compression. Reference numeral 1Q06 or -t > i -f - •> i- Tn 

numeral 2007 denotes an interface for a recording medium, and 2006, a recording medium, which uses, e.g., a fiash 
25 memory in this embodiment Reference numeral 2008 denotes an MP Li; 2003. a work memory; 2010. a matching 
circuit; and 2011, a camera controller. 

The operation of this embodiment will be described below. 

When the operator inputs .an operation such as recording reproduction, or the like ot an image wsth respect to the 
.camera controller 2011, the carriers controller 2011 sends a signal corresponding m tne input contents So the MPU 
28 55308, which controls the individual units. Assume that a three-dimensional Image sensing mode is selected. 

Right and left images sensed by the 3 optica) systems 1 002 and 1 003 are formed on the image 

sen aces of the CCDs 102 id 1200. These images are photoelectrical^ converted into electrical signals by 

tbnCCC; a 2«-«i ct re - -m a r ( s.pc ed to the AD converter 1023 end 1 203 via the COS/AGC Circuits 102! 
tod C- t T M i v ' ^ 

as by he A/D converters, in this case, ssnce 3 « synch r c sl\ 

driving the optical s - dt-- > ed at the same 

time are processed. 

f i q -1 r it 1 C A ^ " 1 ? i % y i u mo J; . 

memory 2003 by the processing circuit 1026 The other signs! is supplied to the signal processing circuit 1027 via the 
40 processing clrcu>1 1026. 

in tne signal processing circuit 1027, the digital signal Is subjected to color conversion processing, and the tike, 
and is converted seta a pixel size of the liquid crystal display 1004, Thereafter the processed signal is transferred to 
the VRAM 1023, At the same time the image data :s saved again :n the process memory 2003 via the processing 
circuit 1026. 

re e sig t t d o c t ^ cr ng d d in tne p m 1 re 2 r - 

is Scent to the signal processing circuit 1027 via the processing circuit 1026. ana is sereectec to desired color signal 

g t c r e d <■ - ec s re wC 

u -ers 5. Tne p seed m 1 -I cd re the <AM 1 023 
"li « . \ j 'C: " rem . . * 1 ; r *j 0 itroL 
SO At this time, the signals sensed by the CCDs are hsd as images in the process memories and the VRAM 1028. 

Inocs 0 generate a three-dimensional nag 3 ' d on the quid crysfc display 004 in the 

binocuisr camera, the contents of the VRAM 1025 are used The VRAM 1023 is a display memory, and nas a capacity 
'o r Uo nar es o< - 1 n - 1 1 cy\"- S'cs'"c 'mc : - *m 3 re 1 

m^ f ! s _ „ a - 1 he so- d 

ss crystal display 1004. tne regnai processing circuit 1027 has functions of performing decimation, interpolation, and the 
like 0? such image. 

The right and left images written in the VRAM 1028 are displayed 00 the liquid crystal display 1004 alternately 
along the time axs »'ia the «cuid crysta ospa/ control circuit 1022 -X; tns time, t~e s 1 , nchroncation sicna J1 " 
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n r c z o \ - es is output from the terminal 1008 as one output 

terminal of the signal processing circuit 1 027, and the liquid crystal shutter spectacles 1000 are driven an synchronism 
.' t i tsj ihronizafcc 519 n this manner, the ct jrve car c ervt 1 «®t dimension* mage 

On the other nana, in order to generate a three-dimensional image signal to be displayed on the TV display 100S.. 
s the contents of the process memories 2002 and 2003- are gsed The ccn-snts or these process memories are 'Output 
as the image signal HOC from toe terminal 8 via a video signal Interlace in the signal processing circuit 1027. At this 
time, since the synchronization signal is sen; to the liquid crystal shutter spectacles 1000 in the same manner as 
ha" t (. - t- -t i « Arspe-* fe s 1000. 

0 i t v h tf - " h -< -.-m s -i >tc tep» 

ro magnetic disk, optica! disk, semiconductor memory, and the like can be used, in this embodiment, a flash memory will 
be exempted Tog interlace 2007 to the recording medium 2006 stores a three-dimensional image signal in a 1 - 1 J 
stasa an empty a e ned» 00 r r " 3 filer gement arsa. This 

processing starts \ the use; inputs a desired operation lor starting recorolng at the camera controller 207 

When the user's instruction is detected by she 7 - J 20QS. the contents r the process memory 2002 am sent to 
''5 theec . :0-::ji7'm ; ■ : _ r _ n * m - - \ 1 Pc " s v - w i ~e n 3 Sic" 

cheme. The omprsssed data is held in w 0 1 h - 33 

arc sent to the work memory 2009 via lite comprsfssion/expansson circuit 2004 "Thereafter, the right and led compressed 
m .srs r f f r r -- -itti'^e c_n Nation -> < r ti r 

for identifying that pair of Images is simultaneously recorded in the hie management, area The camera user array 
ss observe a shree-d vsthout any record 

The flow of the processing upon sensing a three-dimensional image has been described Tee user can stereo 
^ -\. - 1 observe an mags s ^al sem»< a by the ea sr ce the can also observe an Image on 

the liquid crystal display 1004 buiit In the camera main body 100*, the decree of freedom In image sensing is high, 
and the user can check stereoscopic expression ever; when he or she moves while holding the -camera during image 
ss sensing.. 

Reproduction of a three-dimensional image recorded a the 'scorn > < -eoen 

the recording medium 2000 can record a plurality of three-dimension a mage «■ t 1 ' r l v 7 chec<s *he 
rn -e.v-n - i- g medhjn anon f r - ~ t- " ^ 2 

selects files that can be reproduced as three-dimensional images, arranges a list of corresponding file names to an 

38 arbitrary display format, ana sends the (is! as data tc the display controller to display It on the liquid crystal display 
1C04. The operator selects a ^e 5 -0-1 c 0 

2011 The input signal is sen: from the camera controller 2011 to the tVIPU 2000, end data of the selected tile Is read 
out from the recording medium 2006 via the recording controller 2007. The readout data is transferred tc the work 
:.rnemory 2009, Thereafter, the information in the work memory is expanded era the compression/expansion circuit 2004, 

SS ^ „ p ^ <. c -m y o - t 

image data are transferred to the VRAM 1028. and are displayed as a thrae-dirncrisronal image on the liquid crystal 
display 1004. In this manner, the sensed three-dimensional image can be easily reproduced. 

When microphones (not shown} are ananged together with trie smage sensing optical systems, a stereophonic 
efieol can also be obtained for a voice together witn a stereoscopic effect lor an image 

40 Processing executed upon sensing and reproducing a norma: panoramic image as a two-dsmerssionat Image v 1! 

be described below At this time, the layout o! the image sensing optica; systems 1002 and 1003 is different from that 
men rj r x, nag nsing optical systems 1002 and 1 3 set so that their 

optical axes are not gar alls! to each other, as has been proposed isy the present appiloarit. Ac a method of synthesizing 
two Images, a Known method described sr, a 9 , Japanese Patent Laid-open Nos. 67 41237 and 6-217184 is used, 

45 1 1 he > 1 ar came n e state wharem the 

he panoramic im mode at if Her 20 1 n two image he matching 

t e c; i; - omh f r age 

As 4 c e s ma r ' 1 ' 3 upc ma threeoslmi nai image are 

performed ur- ; and is ti Thereafter 

synthesis processing -. done, arm dirferent m r on i cc j'or ptoaesses ate made for the synthesized image to 
be held in the process memories 2002 and 2003 and that to be transferred to the VRAM 102S. Since two images are 

t „ i i k = cvj -Ov r g oc^ o r=^ c c r'^- m- t « - j <■ - e 

educed in the vsrttca ieectio c wi - a i a J in „_> rj c s j < ay 1004. The in gs stored in the VRAM 
1028 is displayed under the control of the liquid crystal display control circuit 1029, 
5= With v bov rm tng, the.user can sans age #hieccnf m -g - 

easily switch the image sensing mode between the three-dimensional image sensing mode and the panoramic image 
sensing mode. 
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<Sixth Embodiment 

~g 26 shcv,= the a - ee nem o* d sy 1 -" T i i f m -"t of the 

present invention. 

£ Referhngtc F;g. 26, reference numeral 1010 denotes a PC display; arid 101 3, a PC mam body; Reference numeral 

101 2 denotes a reader tor reading an external storage medium such as a card and information stored in the camera 
can be transferred to a PC Reference numeral 2000 denotes a selection switch for selectively supplying one of image 
signals from the PC main body 1013 and a binocu isr camera 1Q01 io the PC display "010. Reference numeral 1007 
UK " p ^ n 1 c r ; -i - - p < n^ - '-l. m 2 N 

10 and an image signal 11QQ. Lsqutd crystal shutter spectacles 1000 are driven by this synchronization signal 1101. 

The difference of th>s embodiment from ids fifth embodiment is tost toe PC display IC'IG is used as an external 
stereoscopic display. 

The system of this emDodirnent comprises the selection switch 2000 that can connect the PC display 1010 to the 
binocular oamera muse it as sn external display. The image signal 1 1 00 and the synchronization signal 1101 ara output 
'5 from the b-inacular camera. Ail the time, the PC main body 1013 Is disconnected from the PC display 1010 by the 
t "0 k a i- o i i la r T i e 

seating a three-dimensional image ;s transferred from the binocular camets to the PC dispiay 1010 in correspondence 
With the vertical frequency that can be set by the PC dispiay 101 0. At the same Urns, the synchronization signal 1101 
is output to ihe liquid crystal seeder spectacles 1000. in synchronism with this synchronization signal, the liquid crystal 
20 hut i i bssmes a J n I 

Tr?e flow of signals m the camera at that time will be explained below with reference to Pig. 27. 
Referring to Fig. 27, reference numeral 2012 denotes a graphic controller having a function of oufputfincj three- 
1irt"j ^ " ^ - 

hodiment. except that functiona; pomons associated with 'he output terminal IOCS am excluded from the signal process- 
es ing circuit 1027 of the fifth smbooiment. 

Tine operation will he exptal; red below. 

Sirica the operations until images are sensed and a three-dimensional image is generated m me binocular camera 
are she same as that in the fifth embodiment, a cstailec description memo' wits be omitted. 

Rigbfand ft images get 1 as a h - r ~ r " r • ~ C 3, and the graphic coo- 

1 000 are driven in syncnmmsm with the synch r on :z an on signal 1101 synchronized with free output timings of the con- 
troller 201 2, the observer can observe a three-dimensional image. 

At this time, the vertical frequency of the PC display goes not always agree with that of the liquid crystal display 
a • no rnera. F iso r t i *cas by making an input 

OS at the came ra c on; ro Her 2011 : 

(1) LCD is not used 

(2) Both PC display and LCD are used 

jo Whs 1 5 play and q drsp re - - on signal and the are aches 

1 - s case, the PC d a sted witn respect to !i t - a f 1004 

d tquld crystal d efp r geanps 

synchronization signal 1101 are output in correspondence with one of vertical frequencies that can be used in the PC 

display, 

4S With the above-mentioned processing, the user can sense a three-dimensional image, and can confirm stomo- 

c ■> > - i ~- s e- F _ f I nr t m s F" 

can be set to be higher- than -hat of a TV, the user C an enjoy an image suffering less flickering. 

5©Vs 'h Emdod ■ «> 

SO 

Fig. , - shows the arrangement of a system of a oinocuiar camera according to the .seventh emfaoaimen: of the 
present invention. 

g to Fig. 28, re enumc " - rence numeral 70-00 c ss an 

which outputs an image signal 1100 and. a synchronization signal 1101 from a camera main body 1001 . Reference 
ss -a 01 denotes- a sete r v ppfc,r>g C ns cf inctj* n t 

and the binocular camera to the 3D dtepiay ion . 
The operation wrli be described below 

The difference between the seventh embodiment r the fifth and sixth embodiments is that - - r s 



EP 0 S30 034 A1 



image data is transferred to the 3D display 1011 connected to the »C via the interface 7000 and me selection switch 
7001 , and a thres-dlmsnsionai image is otsservsd using the 30 display '?011 . 

T s f6e - , r > . v n ^n-.r.r cr«9. AS this tims ; in the camera main body 

,q 01 trc _ i - a --•qed m „n ,s c , *m J I — =pc arc ; Out "o 

0l - * 5r ~ „, - , - „ G — - . t It u ^ c f i,cus«-e THu 

tr . fe = - a r v r- t j ™- "1 

J I n M ^ L < A" 

- 0 04 < -is. r <- 1 ~ f * " = - ! 

f0 pr d J P 1 - - 1 -t, . r l II' 

erw ,, p , . r « — a - j--e ^ ip-i V --n h s - » ■> < e mM 

p ssm s2 and - 4 i jess ernaiely arrangsd in units at, e.g vertical 

lines, and are transferred to and held r s work -mr v ' 2009. 

The data ar ranged on the work memory 2005 are transferred to 3 graphic controller i via a - t that transfers 
•s jt r -7 > - % * - 1 ~* aprx.^n-^atlC^ 

n " ^q 1 an- ^ 1 _~ . ^ 

■he same operations as <n the fifth embodiment. The graphic controller /OH cutouts the data alternately arranged ,n 
isofvor ho map, gnai 1100, as describee emc,- 0 ^ - ^ ' m ' 

-nthsgra i. in this man - - mage can be observe 1 h " 1 

20 oonne . , + accordance w;th the signal Jtput rom the binocular camera 



< Eighth b m nodi men t> 



Fig. 30 shows the arrangement of a system of a binocular camera according to the eighth embodiment of the 



present /©nfiOn r g. 

in this ernbedrns; 
camera as displays til; 
embodiments, a reps- 
plurality of synch-omz; 



i r > " images, various types of displays can be use: 
<Nmth EmhGdiment> 



I" l-'.; 



is are not always am/en t 

output u correspondence with these displays. 

Stal shutter spectacles 1000A and 10003 are 

outputs arbitrary synchronisation signals and 



Rq 30 shows the arranoement of a system according to the ninth embodiment o* the present mention, and Fig. 
33 shows a signal otocessing portion. 

Referring to Fig. 33, reference - m ~03O denotes a RAtviDAC <• converts image data supplied from a 
\PV f v 4am c mm "r-jt i2< tc ar arts oc l »min-mm e i * cs _ t - « v utiedsc: ~ oa 
guide ( >i ^i^pm, ^ 1 

The oosratbn will Pa described below 

InT! - bod riant 3 1 * n a FC, a 

j 1 d- > frnrx, o^aPC^ n Ix Jv u ~e ur - Isr 

? - 1 -~ ' -i^t ' - IT - ■> t 

prepares for reception of data used for generating a three-dimensional image via a digital interface tOOS seen as a 
J S 3 or ins like. 

The received data is temporarily held in a work memory 2000. The Meld data is s <k - to and expanced by a 
cc orf -i ex t 3 . 5 ~ p 3 " -3" 5 --iii-'^r^ 

. ^ { 3 u r j ~e t ] a f 

the UPU 3008. Right and - images written n the VRAM 1028 are converted f rom cigitai signais into analog luminance 
t, - f 1 ! t 1 i p t. i < 

^ , »3 im i h..3 '^t .. - 2 - •'• - ' •- m -c o v ^-an- is cetennirted 

b ticnapallet ore r e R AMD AC '3. 

/h n m ^ pt, o ^ s h i ig ( tn c t-cphc 

t- -t di" alsc be SJlo-red t sr a voice together w r a ste f eoseoptc e«ect tor an im^ga 

The gene is ana! tma^e c G n be f6 cofCed in a «w.*ng med urn n ^ccor fen with * =m e 



EP 3 83* 034 A ! 



" ■-■ • 1 ~~ W r 1 - ' -rc^-r' 0" s -pi t » o<~ *e a g t 6 s.ch as a 

USB or the iiks. Aitemativsly, data may be input from the recording medium, and s thrss-drrnsrssionai image may be 
-J ed or he rent data 

Furthermore, with ifas embodiment, even a camera that has a single-eye image sensing optica; system, and allows 
to input a three-dimentncnat .mage using an adapter device can stocess a thres-dimsnsionaJ image 

< 1 0th Embodiment 

FOg. 34 shows the arrangement of a system of a binocular camera according to tine 10m embodiment or ths present 
invention. 

Referring to Fig. 34, reference numeral 4000 denotes a speetaeiehess 3D display. 

-*ecs -Sctowav 

r n i => - a ~~ « - - r 

>' r-ir - - tit ~ 'nm 4 \ i-t d he k 

r - k >£> - n front of the display pan r . , es of adap 

3 -i andao adapter using a lenticular lens, an adapter using a parallax barrier, arm the lure are c ; v J d 
The right and ieft images obtained Inside the binocular camera are alternately arranged in units of, e.g., vertical 
lines arm - o -d , ^ . ^ u ^ ^ ^ ^ v d rssS dimen ic^t mage aaot e 

f c »ct\scrn i i t ) > f c 

separately input to the right and left eyes of the observer, the- observer can Observe a three-dimensional image 



<• r ' J 11 J 1 in H--mt of the present Invention. 

Referring to Fig. 35, reference numeral 2013 denotes a command interpreter which interprets commands for a 
stereoscopic display 

Generation and display of a three-dimensional image on the basis of data transferred from a PC wis! be explained 
os low. 

The user selects a data transfer mode from a PC main body to a binocular camera unit via a camera controller 
2011 , When the data transfer mode Is selected, an MPU 2008 of the binocular camera prepares for receiving data tot 
generating a three-dimensional image via a digital interface 1006. 

The received data is temporarily held in a work memory 3009. The held data is transferred to the command inie-r- 
prater 2013 to generate a three-dimensional image later. Ths com- ' 
basis of data from the PC. Commands to be transferred to tre PC % _ < 

iins drawing command rve drawin mrna _ ! om":n. *i - t -en 

rle'peirjar-'o i u. o o.'m m o;v- ■< « - m;; . ""3 - ; on_ T - trf-um: 3 i mn t., ,s 

supplied to a VFAM 1 033 under the control of the MPU 200« in the same manner as described above, and :s disolayed 
on a display 1 04 via a grapn jit 1023 and a RAMDAC 1 03< 



<1 2th Embodiment* 



Figs. 36 and 37 are diagrams respectively showing the system arrangement and soma! processing of a binocular 
mera according to the 15th embodiment of toe present invention 

- >c " u ^" - - i - J j t li-ai.kt"-; 

display 1010. 

I he different of h s imc jdtrr mt "rom She ninth to 11th embodiments s that a three dimensions mage is net 
output tc q piay 1004 of a camera main body 1001, but is f o disc 

the can - o, tr h< "i , ~ n ~ t f ] c - - ; ir - j vro. di 

imaged jtput to the PC dispiat 1 using th n n tf j a p T <■ ■> p •, 

any function tor outputting a three-dimensional image 

r t'-e sio f j ■> -r> i m -e " i c 1 t 3 ms - _ 

a camera controller 201 =3 befit t rfc iimente. Upon s ? nc he data an e mode, an MPU 

2C0S 0? the binocular c - prepares tor receiving a mree-dimanstoni mage- via s tgt i m face 006 

The leceived data iSjemcoramy naia in a work, memory 2009 The held data \ seopliecnc and expanded by a 

to a process memory 2002 The data sent to the process memory Is supplied to a VRAM t028 under ;he control oi the 
\-p ; 0 p t Q p-jpr rc r v.tM329g»-e aes - h! e-3tnv» rsfonal .mags s\gna' r t> .01 

synchronization slg dIUQi n this em&odr~ent t a camera mam body 1001 comprises a liquid crystal display t004 
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i may not comprise any display 

< ^ n , 1100a Sab. a o ^ , uSjjl ;^ a , nfaM10 - r . 

v - 5 ;<jftnect©ns with the PC disotev 10 

11 1 ** 5 c s liquid csysta u«er spectacles 1000 ir. s r, w ^ 



" -r rt- - - w t - =tl o- „■> A »e e 0 fp all 

1 " l;jf ^eosoop;.: view f moving images in the 13th to t embodiments as k the oresent invasion ;s 

q - > * fem,» r-p <?m^t ~tq « n 

' / r , snoWing " ^~ cf a systi5rn navin 'S a £ — p p ^nsing apparatus st ^- pc 
iU ' ' ° !n 31 ~ rr:S ;:4£sem invention. T system comprises a binocular image st n 

' hr t,v '^ r < f , m t , i T . ng Pt ,sran s a 

a persona; computer with an imago processing function for panoramic v:ew is 



conventional binocuter oat 
arranged. 



, ' ^ f !n ° C ^ r ! m3ge SSR3in3 appar3SUS S00T «™P™« two. gh, and iaP .macs sensing coticai S vstems 5C04b 
„V ^*^™^r*V r . , „ . ^ 

fi,T*.w-«r. * ' ° ' -of? v > isa txJCteb and ©00Sa. and CCDs 

" 3 S e 3« 2 CCDs 6005b and 3009a a re connected to the s\ n* 

siormi generator 6005 :o be able to execute svrmhm^i, ^- w . 2n<; i r . m- ^--v-,.:-. - , J": 

^; ™wy =oniroii,. r 50 i a, an image synthesis 601 5, an mage correcr^/overlapaing amount calculator 6Cn6 
"a wn^ara conned to a CP. BJ . , 

An .mage sionai is input from the binocular mesas sensing apparatus 6001 via tna paraiiei interlace SGtl and an 
image ts output to the aispiay device 6003 via the display corurator 8014. ■ 

The panose m F r 

of „ .is embedment wit; oe described bslow with the aid of f n — c q ^ f? , ^ „ 

s selected and m V md 1 he processing mode inoiuaosa ' .mr " v< 5\ r ; tor sensinc ;— ^ - r . t , - 
r i -r-ocon ,n% uo 

J n " ' ' -a, a'Jn*e<. rear T ^ r 

3 r the p xesshg mode. ARernatively, the proems may be Elected and 

T : 9 fl ° ' ^ - a " * Processing modes W i!t be exotainsd below. This proc^r, 

<Dssp!ay Panoramic lmsge> ..13th Embodiment . 

- * S t 202 to 



ivhentb^ too U "' r f » t ■ 
W e" :h, < k ^ v 

to sense ngnt and left oaratiai 



sssing in steps S120S to StalS is axecatao: and 
' ocLited. 

iontctheimag ;t sing apparatus 6001 
ano 6000a The right and leit images are 



synchronoaslv acqnrod on the oasis of a s^onat Smni mV'^^o',.^ 
apparatus 600t convents the K Q ,^^ c . a % r ^\^ ^ ^ 3 > r ^oa sensing 

- r ^ _ .^g^ ^cn^is py .as A/l; conveners OPOSb and 6006a. 

' - " " vs - C rnnnermore, the image sensing apcarstna 5001 inputs the stored 



and stores these digital 
image signals tome pi 
the memory 6013 via t 



trans; erred to 



Q : ' CDs rtchajriousiy esscute ffitage sensing, a isloc-tc-d 



image signals a.o image transfer from the binocutar image sensing apparatus- 6001 'to thai ons ona corner 6002 
«re .otsvnch , a v p cmett i^e* ro- o s - !f =4S 
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each other , . . K ^ 

iefi transferred psrsllax images in steps 31 SOS to 312GS. ♦•„.„*, 
Fq 4hsbo wm^tiadof^pm. rqa^o s~ 6 vi hesi« maga r lie r J 

- - <. i e <= — 7 ~ « odd , { r - f 

, r j - t £ t l8-e p - r c rr Jt . ,m i m " < 

, , ~- <, a v ntrm c n -*-d i 

,h r; f psxais ft - ' i s an "overlapping arnot; a this embcdime s sedan omh 

tes ,o^ - '-t ^ - ^ » " n i , - J 

- ,m 4 t t- l " - f c 

slzer £015 in Fm 3d bsmrmmm ovar!app:nc; regions cf tha mm right and iait images 6201 b ammmOla on th* oaafc 
v. >hc^ \-U.*> n ap °1 - J d r 

c h it he ) f ! a of the nqhi and sert 

lagss into the r r n « mo images 6201 b and 6201a < 

hd^L<-J"itt 3 V t a ^ " t r ~ 

\ _ r- " ~ 5 " jn < " h - "t" c a a J " 5 r " 

i r t t r \ ? ^ ^eiemmsoov- t - . - ^ t - an;- arooa orarrtc 

synthesis processing is performed using She i value. 

<R«cotd Panoramic smage>...t3Sri Embodiment 

The processing executed upon selection of the recording mods wili be sxpiatnsd below 

Fin 41 shows She panoramic image senemg/dsp^ mmhcc in the recording mode. When the recording -nods is 
iS ^ ^ ^ ir. ah 

; f , 3ens9d - fn ma same manner as in me through display mono cascaded above m the wording mode as v,m, am 
4, „ a , n ,- 5^ , c a. - =. s cm c -m m fm t m d spiay mode That s par 1 ~ . ' 

Y> t - m \» v " " 'd" 1 riaca 

i ^-"tne two r.ght and left images 6201b and 6201 a into a pancram mag on 
i M f m„ H M r-o ^ v ^ 

j ?p *yed on the j sp ay soparatt % S006 r stao S1211. The user can record 
ij, 6003 

' \ - ^ , r _ Is v. always simple unlike n Fig. 40. For example, trie lumsnanca of the ovsf- 

c i i a a' - t - " a a r t J -> " vr " " 

, 4 ^ n o r , ^ st - i-j^p.-m^ ^ nia^r m,enjx„ 0 : ru 

d - o t M- - disglav nmaa, the - :magas 62Cib and 6201a are directly > h ^ z by assigning a prsda- 

M d - o»s ^n ^ !a-^ j ° ^ K T s ate rr r ^ c - h.r 

1- o -, - 1 - f „ h - ^ro « " h r ^ 

i -co m f f c=.pr mamoL-t ^-c.e.. sr-i the record -made 

- 

In ssap 31215. ihe mage caaecdon/ovadspalng n calculator 6Gl o caaects tde sensad i e t 
-rs ^ ( . ih- c 1 c = mm v ^ % '< - ° 

ir , i c m f -d i vr 

^ h m a a - Jn j m a l 

t ♦ n - - s . - m PP'^-' 

the image synth&sizer 6015 synthesszss a panoramic image in step SI 21 6. This syndesis s subaranttaliyjna same 
s'hat a ^p 5 " t 'au'c^' d rt ^ " 
parameter between the two, right and teft images. Also, tns two mi 
image corrsclicn In the recording mods 

The aanorarmc syaaiesnmd :maga is t-a rd to and recordea 



en subjected !< 



■ 2evsca 50t7 i ep 61 217 -n th s 
Ts setting, in step SI 21 2, it Is checked 
d 



ti ddoiay and - d -c am 'a ena. ir \j n step - „ „ ir=3 „ % -»j > = 
end step S 1 213. m this ^mbod went, re panoramic syn ativeiy : l ^a 

edtmagas 3-d hecveriapc g amoan as ait cuti rtonnation ma be recorded 
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Reproduce Recorded image;--... 1 Gin Embodiment 



i upon selection o; the reproduction -nods wiil be explained below 
selected, a fie of moving images stored in the storage device SO" 7 is read and is 
5003 tn stepS-C 5 e hs mse mages e e a ve- - ^e \ XZl ^Jsr 
5014 in Pig. 38. and are displayed on the display apparatus 6Q03 as In the asovs- 

srrasponding to the user's purpose car; be dene by switching the: modes. In the 



- 1 J 1 " f i - C old. 

ibieSGlO or 

connecting the binocular Image sensing apparatus 6001 and the persona! computer SC02. two. fight and left images 
n - f %- <■ t t5 j r ^ "(» i | < 

ferreo to the p i| j ^ nc tebace. in actoit IheCP! his ' 1 tne persona! co uJj 

6002 may adopt various types of buses such as an ISA bus, PCS bus, and the like. 

<14th Embodiment . Stereoscopic View 

The ie snmcoimem of the present invention uv;S; be described eslow whin reference to Figs. an an d 43. 

The above-menfsonsd 1 3m embodiment obtains a panoramic synthesized image, while this embodiment obtains 
an i^aga' us it s 

embodiment is the same as that shewn in Pig, 36 in the 13th embodiment described above, and the tew chart of image 
s-nsi g-cLpd p ir r \ [ 1 L -sir ^ -g rtp„ ams o J ' n 

embodiment is the same as that shown in Fig. 38 in the 13th embodiment described above Hence, the following 
explanation will be given by quoting these figures. 

in step Si 201, the processing mode is selected and switched. The processing mode of the 14th embodiment, 
includes three oxides, i.e., a "through display mode*, "recording mods*, and "reproduction mode" as ir; the 13th em- 
bodiment described above. The user selects and switches a desired processing fom * n n f < ~o i , - icdts 
t t n ) as wall, Uie mode selector 6018 as a hardware arrangement in the personal computer S002 in □ 
' s mo t-P p cwssosm /- ■> - - l j le-ted 

and switched by software 

-'.Through Display Moae>, 14th embodiment 

st oijgh Jisplay mod elected, t* righ end iett nages re sensed in step St 202 The method of 

- f f Tl v L . r C 01 !> ti S ilL sS 1 - ! u J 1 ' 

T < t5r na J jr " c ^ ^ ""m r i rsa^ a n * ( v 

>i wti i • t - i 

sensed from two different vsew points. 

The Swo, right and left sensed images are transferred to tho personal compute! 5002 to generate a single image 

F ig-4 4 :o 3 3 0 p:c the display cpdra $8003 hem o oh 

1 i;' F- .2 -e f H'« ce sun 3i t a I ge> g , nght paraiia 

mage j f p i ^ N a - aqs for stereo 

from the two, right and left images 530 ;b and 630 la must be determined The overlapping amount in the image for 
-So ,omopi-we-. 'dvstei- -mc - % " e • -ot eft :magss are mao ; - rio^a h h r .m - , 

r u smo a pgrailax of t - 3 soopio view i.e., s on, cm i 

controlled. 

This overlapping amounts set at a predetermined value in sfepS1203. in this embodiment tne overlapping amount 
value is set to reduce the parallax of the principal object 6302 in each sensed image. Such overlapping amount is set 
o allow sas s oof -~aecso ; hapr - 2, beta > bet esf> set Sythe ^ser ins ep ~'2 n he image 
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svnthesizerSOIS in Fig. 3S generates a single image for stereoscope viw oo the oass <* to sat vato ^ 

Umm .^-waving an image tor stereoscopic wew ; as aescnbed in the paragraphs m h-e pnaa *it. a am,, 
^tn^m-iv outputi.no riant ana lent images onto the display appa-arus, and observng the images v-a nqnaa ury^m 
Lt ^edoeies mat cerform 1 r - in synchronism with 1 switchmg things of me two image,, c 
, aiiemately arrange: two. right ana > images in units of horizontal iinee placing an , , t tna.a^- 

p:; , pj v manges t auction of „ Zdt o - units of lines, and a g ' .mages via ^ \- ^ - - ^ 



The lirage ganeaatac - s manner 



is transfer c r s display o 



w , -.anaar as in the image q „ for panorama . % . - above, and is o^spiayed on the display 
^;;V, : 5 :n g S-aOs'ln step S12C6. images - f - ' to the time and the number or frames ,r by me 

: " ... ., . .., : n 

. , , . m h.ran^r^^ssns^playfor 
sterecscop < ing age f ar - a's,ar-e ealaed 

<Recoftiir.g Mcde> .14th Embodiment 

rae ~*«ftina executed upon selection of the recording mods wit! be described beiow. 

^, n - < x ie « ^ - n M ^~ 

t ! t " ~ , Imeae 630 tV ''a' righVparafe inasqs: ana 5302. a panada! - V ^ 

h M . t.r r . 

agnate « the same manner as;in .thaaboyw«tk^.thfQughd5Spl^ 

m0d - fi 'he recording mods as well, display in the through display oxide » dcre en flis displa if JS S003 More 
^ f f c< slocu. \ - ova^s^cr.ig ^n^"' is $' tat " 

/n ties ?er 6d5 n F,g 3S symt - - s 

je ^asvn< K^m-, " 

"7 ; ; , VRAM , v;i?n bv lhe d!ap ; a y controller (Fig :Jet and ;s displayed on x display 

• , < . a an mage tot stereoscopic vewwh-s chewing the image display 

c an d oia n 3tus S003 . . . , . W Koesri 

. if , -^^Hinq p . — - svnthasized irnaaa. wnen an image tor J varw i g^nsro^., 

1 c ^ r chdn>ca\ -rode, the - ^i.ort, vr 

r 0 > > t . - r ^q < * 

~ £ Z , , -.r view is aanaratsa. rhs imags %-,'C'-'- -nq amount calculator oOio i ; r , ^ - 

^ ^ i e, - ^ 4 a ^ -rpr-gm. -i 

two ' nfl H ; aric i,ai !;T;ages SSOtb- and 6SG1a : is set by extracting tda Images o: Sna principai cagec; 
a ,o ^ jr, - - 1 1 

3 mc ndicates nt the two. not i 

t , , , , . r . * c~ ^ t f m<. n a- j 

s;e ^ c 3 c , „ Vv . v _ n ju value in 'his embodiment, the parasax ® set at zero, but may oe se. at a v«iue . , 

thanzerc ■ M^asWmw? io ronv 

in step 81215, the irmaga .orrsci^overiapping amount cainuia.cr o,.^ nona,^, ^ e ;; r .;;:; f , 

pensata, tor tha lanananca ana =^«^^ c ^. t «; ; ™ ^ ^"orr^ac^ 

it t - - - — -i-.tf^v- • - ^ ^.- w;lpri 

rr;e nght arid t oarafiax imagss 530 ho' ana 6301 a'. Attar suca correctrans anc r t , 

tna ;maqe synthesizer 601 5 (Pig. 38; synthesizes an imago tor stereoscopic view in s ^X,^>^ 
su-s -mtto m -a "5 a S t : • s'eo S ^ n - 

a - f m t ^ < " ' " " 

, t -, ng mode. 

-t a -a » or^ - ->! 

r . j ■ - -t.-atsty of frames are recorded in correspondence v:h user's 

/ and recordr-g are to end it NC tn ^ep 51 21 > the flow -et j rns to step 
" :; 'nstepS-2^3 

, lv ^ , -a e 2 * =n,q - h\o am on. ■ .m 
tma.es and the ovadapping jncu - as annbatrs information may bs recorded. 
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> 14th Embodiment 

F natty the pr<x ess-sag executed upon selection or ins reproduction mode wHi fee explained below, 

s reproduction mod act, a moving images store he stooa \ D F:'g 33 

read and is displayed on the display apparatus 6003 In step St218. h - case, images are transferred so she VRAM 
6020 under the centre; - the display controller 5014 m Rg. 38. and are displayed on the pa apparatus 6003 as. 
in the above i gfr aaj mode 

A« dew. cm bo e 1 Ti ^ - 

<; ^ , r e n~ , . r 5 2 ' -Ovfg "caqOb 

sensed in {he through diselay mode. the processing time per frame can be short, and moving imager; with a high frame 
i can be presentee in real time. In ;ne reentering mods, since an image for stereoscope view ts generates attar 
- -j correction ana calculation or an appropriate overlapping r are eons, when the recorded image o amplayec 

can be obtained. 

in the 1 3th and 1 4;n embodiments desenped above, the through display mode or recording mode a, selected before 
l3 , r H i - n r c kvJi co . „ « rj aat-s •"•><< d t Aj* ativo > whs 

the mode selector 60 1 3 turns on/oft the recording mode from the through display mode, a processing algorithm that 
performs synthesis and recording of i age parall to the tl ugh d l ode of the display af rtas 6003 can 
be realized. 

Uoon irnaq n P ge s t s are set uair 

r t n > t . it - ^hn ^ i h - i - % 

- » « i r > -m> set p-t'dilai to each other to be separated t - — '-mo. „- s o.a\ fc 
-t«r.oa >i , < e - f ^ n "L - 1 > n^r s - - ^ .astngir 

t - - ja ^ xco - - r , - - .^^,^~ nr n .p~ f f Mnr\n 

and three^JirrsenslonsS image sensingrdlspiay for stereoscopic view. 

<15Sh Embodiments 

The tSth embodiment ot the present invention will be described below with reference to mgs. 04 to 4?. 
In this em.ma -»-t acm'o '-,.,> c»p«* -oiec ,n sensed images for stereoscopic view is reduced to zero 
jj 4 1 n - » i <* cec di tan.e of intgo sensing optical systems, thus allowing the user to 

Fig 44 i blocker gram showing t 1 aemem of a brnocula ge so jappa - n . t e 5 t 

instxscfT'e 31 tn.-p 3 ' n ns>o.n tn F g refe ce numeral 6700 denotes a binocular image sensing apparatus, 
nh. h ctrmcrsm \%< jt ciaOim^; .-m a or ,3 s ooh'' . . 6 'C . "^J* 6"h4 lf 

objecs position d tor 67 a find 670 an interface 07. Images ; rsm d 6703b and 6708a 

\ ttr. mc - s t tr " nr-g „ i 3" „ - ^ ~ * ) k r 

^m sc8 670 xattai g inter n ween the PD and the image sensing apparatus 

The image sensing optical systems t b and " respectively compnae lenses 67G20 and S?02a, and CCDs 
^ 1 c ^ t " -1 d ^n - j ji = -r ec J > - " e7--" 

-ni t t ^ > - o'c^essa- 57C4 ar * n - t _ ' ir - 

< 3* : c 0 rsj - imags oatpu d the lih The signal p r 6704 nnected to the 

c „ ! t >r .jtc^t.'O-'e - - i'>o c"m id' r 1 ^ "a'" r f t s 1 . 0 0 ' 7 mn. ir> 

subjected to the image correction and symmssm and the user sen observe an image for stereoscopic view by looking 
into the finder 6706. When an image is to be edited by an external apparatus (not shown) such as a pemona; compater 
je- D t ? „ t t * " 3p-a atasvia 

the interface 6707, 

The object position detector 6706 has a calculation unit for calculating the depth of the pnneoa: object, selected 
^ p^Fr-" t * c r 1 ^ rm m c - t < f 

the pf in - jet at 7,e 

Detection f he con stance n e by t e stectcr 6705 will be explained below 

fvtare specifically, the user designates one point included In the object o? interest ushg the PD 6710 such as a 
rcuse , - - - 11m >~ ( -ant- - c"u6 T 0 detector 

e a s - r " r q f 4 ~ m - tr f 0- ? A " - t cerr o f ^ 1 

c - c t = ^ 3 "-at 00s tfor andc^i ^. te t cpa 

P th from the tm.ags sensing n re p t x he d 

0 s^-i j - i -c A^sn t^e oorstfoi bas^o c v 0 'he convergence angle is done to set 

the parallax of the principal object at zero. 
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Mots that the method »ie 3 the principal object in the image is not fern tsc - f g tne PD intertacs 8709, 
but the prjflcipai object in an image may be automatic-ally extracted. Alternatively, assuming that the orincipai object is 
located at the center of an image, trie centra! pains at the image is determined in advance, and the paraiiax of that 
portion may be adjusted. 

5 ' A * pal object 5094 c v - - ge sen * r 

6701a in Rg. 44. When the object 6904 is sel dsd by the object position detector 6706 i ebinocui - i 
3 i ,f ^ rot " - r - - ir-d ■■ - % to^ 

6701 b and 6701a is detected, 

fhe ooject oosit:c iiitector 67C5 sends the pes n or mat lone depart c Dject £ to the imagr sensing 
m ~cti - - 1 ■> - " i -* " 

- 1 ~ - - —-u bn fc -tl,"-- ->"V b 

570ia t as- ^ c f n i.e of the p pa b -"rb Q C4 

The convergence angle control and f a ec-ntm; or i image sensing optica! systems 57Glb am 

5701 II ex a - del < 

ts a - t u s t - i - n i c k 4 1 " 3 > arailei 

j ,. tk„r c > <~ i -i a t J i v f 

d * oase 

distance >:. as shewn in Fig. 2, is calculated by equation (1 ) baicw, as mentioned previously: 

e=arcta«^ {1} 

When the image sensing opfica! systems 6705b and 6701a are assigned a convergence angle that equals the 
angle S, the parallax of the principal object 6504 can ft be reduced to zero However After the right and left image 
25 se r , me e -d -7 a are respectively assigned a convergence angle, sines they have different 

conjugate planes SSOSo and 6905b. a distorted image is formed in the peripheral image portion excluding the principal 
ooj t:', 1 As the ;c . '.ence angle Pecomes lager, this distortion s becomes larger and n becomes harder to 
obta:n a good image for stereoscopic view 

Hence, in this embodiment, the convergence angle assigned to each of the image sensing optical systems 6701 b 
■30. and 670ta is set to nave a limit value defining on 0 allowable range as an image for stereoscopic view and is not 
increased seyona the iimit value, 

As the limit value of the convergence angle, a value based on given setting conditions or empirically bra 
i, it-, N- " i i j i ■ • c gence angle is held in a memory arranged in each of the image sensing 

opt J \ -fan 6706a ) n in Fig. 44, and it is always be checked during driving the opitcaf system 

as if the convergence angle of each ot the -meg- - ,- ir ? 0 - : ca . systems 6701b and 6701a has reached the limit value. 

r embedimen at tanc e ie ~ a i a 7 r,d 5701a is set a 

55 mm, and their convergence angle can be controlled to reduce the parallax to zero for the object position range from 
Infinity to 1 m. That a, In equation id) above, g<ven 

~° S s 65 mm 



/. ••••• 1 m 

45 

the limit value ot the angle 3 is 0.0025 had) m 1 .66 s !, and this mage sensing optical system can have a convergence 
angle up to 1 36*. 

The method Of controlling the Image sensing t „ > systems 5701C and 6/0 1 a to reduce. So zero, the parallax of 
the pr:nmpai object 6904 when is located 1 m in free! ot the image sensing optical systems 6701 b and S701a will be 
so =k s e art m Fig. 45 

Fig 16 si 3ws ti ■ 3 v of the >■> - - ptical systems 6701b and 6701 with onve per c^ ang - 

i 3 Fig 3 represents the tin ue \ * er - 31 
In Fig, 46 the object position detector 6705 detects the onncipal object S9Qa and detects too depth ;t from each 
a; re mv,-: u ' 9 " I a- :\ a <n step 31601 in step 31800. the convergence angle 9 required 

SS *cr edu-n i tf ■> p-> - c me J m cv em * " i j or 1 econv 3 d 

based on the depth 2 detected -n step S 1S01 . A: to in step Si 802, the calculated convergence angle 9 is compared 
w:th the limit value Q lM of the convergence angia held -n 3 memory (not shown) of each of the Image sensing optica! 
system drivers 670SP and 6708a in advance to check If 
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and if the paraitex can be adjusted to zero if the convergence angle required for reducing She parallax to zero is smaller 
then the limit value e^, u.< if 



and tf the parahax can ba sdjosteo to zero, the parafe is adjusted to titsrc by changing the convergence angle 9 stone 
in etop 31SQ3. Irs step SlSOm The adjustment ends 

On the other hand, h a is determined in step StSC2 that -he convergence angle a required tor reducing the psraiiax 
to ?ero is larger than the limit value 0. M , i.e., thai 



and the parallax cannot be adjusted to aero, the convergence angie is changed, ;n stap Si 805, until 



It " m converger g t jiue j and he 

< ^ - c ^ s~ ^<5rc570t2tcadjus m , I < - ^ n t - ■» > 

Assume a an optica! > L of the -eft image sensing optical system 870! a crosses a conjugate plane B9C5a 
of the -nage srs 1 m .„v up p 1 i i v - 0 A At this time, a distance d between a position A .of the 
pr Irtctpa! object 6904 and the position A" is catenated by equation (2) below 



Hence, by moving the left image sensing optical system 6701 a by d in the direction of an origin Q. .. the intersection 
A" between the optica? axis L 3 of the left image sensing optica; system 670ta and the conjugate plans 6905a oi the 
image sensing surface can be adjusted to agree with the position A of the principal object 6904. 

' * ■> 1 f c <■ r a 

: ™9^ sensmg ophcal system 8701a about the ong:n Q ; . the ogbmmage sensing optica! system 6/01 b is similarly 

V- i I q ?a - 1 \- . m-i»J hshr" " i<- cm ' _ u 

^ *" *j ' f t r "nrccpt-i " a ' m- n^ c I > the 

I d to zero fc the convergence angl - parallel!'/ d 

placing the opticat syste s s rmn the base distance € . 

As desc bm ib_ m h i or f ^ v q r : ^ 

670 1 ha i nci3 01a :s ! a _ a^mn-i 

and upon adjustment beyond the - value C ^ the base distance « between the two image sensing optical systems 
6701 b and 6701 a is shortened in this manner satisfactory images for stereoscopic v=ew that allow the user to easily 
fuse images of the principal object 6904 can be obtained. 

Th meat is ocected tc paraliax r of the z + " + ce H * e presen 

en n t d tr t- gmictt-cm • * <- c m f - a w ^ •• c 

miSth embodiment* 

Tne \ 6th embodiment of the present invention wtii be ossenbed deiow with reiarence to Fig 46 arm Figs. 4B to 50. 

r tr s en d m t> ~ v < c o cm eric c v ;'.a ~ .•: • j c i 

i ^ c ^ r c cn mace sensng optcal s o - - c .. -* <fc v a c c mm- „s c n 

part as of tne pra cipai ooiset in mages tor stereoscopic view to zero, thus allowing tne user to sasily fuse ims gas, of 
the principal object. 

Fig- 46 an a sinocutef Image afus according So the 15th 
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amdodtment of me orssent >■ < s ^ The same - . numeral in Fig. denote the same parts as i, ' ^ 1 

~' ir ;hs 1 Stn embodiment described above wnen the :orvsr S «ncs ang.e axcseds .s :n.* value 9 LM the convergence 
PC h t jt i w i it r 

and S701 a Is shortened io control "no paraiiax. 

, r , t - < i * .-v- j-c a t ^ , h- 1 -v a- 

8701 hand 870 he CC s 67 * and 67 3 - He dispiacec , he CCOdr srsS Oba s 3 te *du 
the aaraBax f the p -;c at set 5S04 1c i'sro 

i •> 1 t t r I ^ - t n ^ 

vitht aid 3 w chart « Pig. 49, 

i step S220t, the <- position cetector 6705 detects tne principal object S9W and da; acts tha depth - from 
I jjnaoe'senssng ptic r,s 670 S?Cta - he cenverg }le 6 required 

~di i - -a a tax doom era J -dojcc » c "4 - a.*' 0/ mmi - 9 t '> f-e - m "m vd -it oU^j 
, ~ j j > S2. Ji < i. C! - 5 " w i" the - it va ue 7 LM of ho x , y C i ir htl J 

r r r — tern drivers 6708b and 570Sa« so as to check j? the 

paraiiax d - be equated to acre - 6 < 8 LM . 

I* 3 :onvo' ' r s i on c a" 1 h _ h< ; b v. c - - -a " „ 5 ^ 1 r 

the limit value a I. e f9< Vt h " Jt c " cr t«ax can be adjusted' to zerc he paraiiax :s rac ed zero b\ 
ccntroiiingViS convergence angle a alone in step 52203, ana the adjustment ends in step S2204. 

On the other hand in some'easas, the required convergence angle Of each ot ths image sensing optical systems 
701 band 67 yd gerthaath* - slue If the convergence ang^e 8 required for reducing the paraiiax 
^ , t t v i <- 4 t n - 

angle a is increased up to the smit value jy, in step S2205, and the paraiiax is reduced io zero by paraiielyi displacing 
ihe'cCOs 6703U s d 6703a Py ^ CCA tSdve 671 0b and 671 C m id 

ru 4* h 'o J tf - orrv -! x 0 r -fu i"CD^dio[| -< r . „ p « 

system 6701 a is teased at a point d. A case wifi be examined below wherein the image of the principal object 6904 is 
»<C ait -e surface ct the CCD 6703a. let v fee the distance between the far-side principal 
plane of the tens 8702a and the CCD 6703a, and x be the distance between b and O,. Than, x is given bye 



x = V- tan(8 - 0 L m) 

= v-£ - 22-tuan™ -f t-tan$ m 
2?, 

Hence, py shifting the CCD 6703a by x in a direction perpendicular to the optical axis Lj so that fae point b ao- 
q\ .a c : ^: i 

Fio. 50 shows the iavout ot -ha image sensing opaoal systems S70f b and 67G1 a when the paraiiax ot the panctpai 
_ — t " " d-'iriccnp' no ~ e - < - 

L, and in directs: 0 separ ri the c ginO,. 

<Advantages of 16th Embcdiment> 

^ described abovs he convergence angte 8 to be ass ig i ^dio't = >c " s ngc ^ii^se*s 
r a 9 w "hv^Le , a - - t '3 ^qe 

^ ^ - r 3 n<- a • , ^ -a \o ^ t trir L Q- r d: "tis 

.; j^.s, j . ; . ce; ,,ceci in a direclon :a a..... ac .: • • : t qt . c ^ . t a v r i 
tj c h =i at. u ' - ' s. c s ^puv 1 --a 1 t- e-s 

fuse i'rnaqes V ths chncipaf object 6904 can be obtained On tne other hand, in parallax adjustment by only paratlei- 
disgiacemsnt of t?e CCDs 8703b ar*d 8703a since f>e paraiief-dfsplacemsnt amount becomes large, Its control is hard 
l m ; t t-^spara ei-dtspa :nr 1 . -a caeth^ \. tr tn* 
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" r i-ercerc ingie control, the toad on the paraftei-dispfacement control can be reduced 

•" ; ;sc"to- -r. ; a •:• .:•:: . a, c,...h: - C : . cmca .:s • • . •; s !^ t\h e •- t-t 

described abovs. Hoover, the present invention is not limited to this, bet can oe applied to ?, case <m t i the oarsJfax 

is -adjusted to an srbiirary panadas amount 

Furthermore, the CCDs are used as image sensing devices However the Dissent invention is no! limited to such 

specific devices out other image sensing devices may be used. 

<1?th Embodiment* 

The 17th embodiment Of me present invention will be described beiovv with reference to Figs. 44 46. and SI , Note 
hatth; '"nr.," ^ j , 

in tne ni- embodiment o. - :ec aocvo and -v be explained by quoting it. 

" ^ ° - hi - K 1 n r-opf.al 

systems 6701 b and 670 = a is changed up to the predetermined iem! value St M . and the control of the parafiaxbeyood 

1 1 itrtt ,;1 ' ^. P "0/1- jM. 

' 1 M ^ t t - , cto sy ) b UK c^' 1 

1 f " ^umnrf 
stereoscopic view from two, right and left sensed images are paraileity displaced !o adjust the parafex of the image 
tor stereoscopic view 

F:c. 51 -s an explanatory view of image sensartg/dispiay for stereoscopic view by the control o? the convergence 
angle 6 up to the limit value- \ u and software in the binocular image sensing apparatus according to this embodiment. 
In this embodiment us we:;, trio irmo.ge scmsm.-g echo a systems 670 1 c arid Cor ;! can oe assigned a convergence anole 
up to the limit value G m determined in Fig. 46. 

As has beer described in the ISth embodiment above, the position o! the image of (ha principal object 6904 on 
the CCD deviates by x from the center 0 2 of the CCD. Hence, (he positions of the images of the principal object 6904 
cevtate by 2x from eacn other on tne CCD surfaces, it this deviation amount 2x and an element size s of each CCD 
are known, a deviation amount p (pixel) in the sensed image can bs calculated by the following equation (4): 



P- T (4) 

The image sensing optical system driver S?0Sa {Fig. 44) calculates the deviation amount x and transfers it to the 
Sl 3'- c ' '-i \ - 1 v ' n y ran ed .i •> fno.ntoi r ^ -m, d image 

Tn * ~ 2 ( " c ievs syn- 

thi-w x n, ^'m r,.., f , , i J - c o> it c 

7401a in Fig. 51. 

- *d the use 

observes these images via liquid crystal sherry spectacles tna; switch rich; and left shutters in , t n the 
11 * N f ' 5 1 h j I r s > - T - - t r v - p *•> - 

* N > 1 > - c> i i it- e , - a p ~„ n ng ' v. 5 f a- j 

= r d it q „ t. % t i - t- i j , r d r t je 

imago tor stereoscopic view r units of horizontal lines, the two images can be synthesized to nave a "shift of p (pixel) 
to generate the image for stereoscopic view. 

Advantages of 17th Embodiment 

vv " tf rara,-v v . - rrsJ- * a,ueo J the cor vcrge-ce ang s s to be done, as described above, a 
" 1 " ^ " " f < - c n j n 

f It pic via cing the parallax of d ^h - o t h=s p 

ir-ou^' m . cj « t n .a s „ r ^ -f L c t » - 4 e - 

without any hardware load. 

~" 1 ^ (t b m^ c ~ r- oJ r 

' tMf ( tpc e - s steJ to a" 

arbitrary parat&x ameunt Aooordtng to this embodiment, evsn when an image tor stereoscopic view ts generated and 

" 1 ft sensed imagss recorded on a mem ry, It can be disc. I , t 

- J < 1 " 1 if 5 P c - S«)4 sewing ae-«je.ob|ecttcSa saised Ihroughcut «ie thitd to 1 7th 
5 Ra on bisscttx between the gnf a id e'f mage isen, ng optical systems 5701 b -nc 
8701a Howave; tne parallax of even an object present at an a mrtrafy position within the image sensing range of the 
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binocular image sensing apparatus can oe controlled as a principal ob=aot 
■mSth Embodiment;- 



i! of the present invention t be described below with ^ to 1 44 and 47. and Figs. 

T nca nsnt ' 1 1 fins ab 

5th embodiment described above, and will be explained by quoting it. 
embodiments described above, when the user selects the principal object 6304 in the sensed 
i n . r - >rg .? 5 m> - ' ml 

the user controls the ;rnage sensing optical systems via an interface in place of automatically 



s a binocuiar image sensing apparatus with a user interface for cow-genes angts/paraSiei-disptaa 
Re^emnq to Fig. 52. 'reference numeral ~$cr denotes a binocular image sensing apparatu 
solace, a finder 7502 and a control button 75C3. The centre) button 75G3 indicates ins P 



ment amount c 
which has, on 

interface S70S . 

- n ,_ fSPnc 3 m . - jn » )H --jc " < -i J dm * S7C9c 5ndo7!J8av3'ne c^h t ' i 

, t c | t ^ q optica ma t d 6701 a. The rent c ** 5 I 

jle a~d ore ng t- s so^troi s?uttort 7SC3 e irn s g optical sys- 

tams g70ib and 670ia can bo moved. 

The ff-H c 'I - 01 ~ r c tt 

75G3 will be described below with reference to the flow charts in Figs 53 and 54. 

In this embodiment, when the convergence angle is bcu c o -xc v - . 

rt: rt i? l -1 * - T " - - ' nc o - x 

T- operation upon pressing the control button 75G3 in the {♦) direction will be described bsiow r the aid of 
Pig..S3. Fig. 55 shows the operation slate of the image sensing optical systems 6701s and 8701a when the control 

sensing optical systems 670tb rand SrCta is measures anc is comgateo witn t i - r t m - f 

,e the current convince a^ge % squai to v the tr^cs 

,\ 5 l ^m -t ban n 1a are roved to increase their convs^ ^ 

hcvv hs ?ta A-nereit e two in snsing pttca systar c S701b k b?01 vitn a c Jnce 
t iimitvaiae 9 ; M have moved bom tne solid line positions to the broken line positions by increasing 

' - -«sed.n the W diction, it is checked in step S2 

mm „ ancle a a; than time o eacn of the image sensing optical systems S ' c and 6701a s equal to or smaller than 
}r n * ? > - " -> n- . k ^ r - c - c p ' ~ "r: " 

, -g a- % li - ~ -> > f - - t ^ f t i re c Hi* f 

0 J L L r cc-kd-re ft d^ to S2S04 a shown in ^5* »b . m. current 

t- j - t , - a " - < - m t p o 

^ n l~ ^'ir^p.^^ m step S260 1 In this case, it not 6 < 9 U < out 0 > - holds. the flew skips steps S2.602 and S20S3 

, r i - acVaer » v mac t - i 5} p ^ * c 

\ . i r -rdi uSa^ f I i a„ , " ha - i mm, 

r vS t"m 6 owj',va<ic is t"t mage se 

j r ^ „ „ x - - - (. t'f S.\- b 

, , t , n c - n 1d - c , o u - ^ t ft 

, q <, h o 1- 1 j ov. mce b )u -ccoa.mm- 

-< d ^ '-e ± ,o • m--k - -in - ■ cos t o's 

I T - o" v ~ <- " r q r - * " ~ ° 

i " } r£\cn + ~ rs r«t - o 

d:<miaced m a direction to gradually decrease the distance tnereoetwsen Note teat tne above mentioned operation =s 
sV - r , j. }e ct.a s.ssni^ t- o- - -lb *d ^n- v e%fv v es the rV 

Va "Tnl oporaticn upon pressing the control button 7503 in the {-) ^'sctscn will be described below with the aid of Rg 

Fig 35 shows the operation state ot the image sensing optical systems 6701 b and S?0t a when she control button 
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eta the state 
n the broken 



- S ,:: r ■) - < ■> .1 0 m„ 1 r 

betwess t 704. sm I it «s deterr to step $27 hat ha cu e snve - r,^ o each of the 

o - * : m. „ t v , - v k 

ar-* d p 3 i _ 1 

it is then checked m step 32705 If the Imaea sensing optical systems £701 s and S70 la have reached {contacted'! 
a point D as the imit point of psrallsl-d'rspiacemar h 7J t s p _^ 5 p 32704: o *, 

l* m W ru -it - -5 5i - i ^. , ) . f r t. X r ^ c=.- J t i . t 

(i Mthti , -r ~ a n r j j p_ , 

to the convergence angle r 0 have been c 1 t > displaced to the double-dashed chain line positions. 

n f manner, when the control button 7502 is pressed m the p) direction, the convergence angle of each of the 
f " -i i when the ca \e B Pr t a 

has reach ge 1 || it 

r^i Te ' p T tt i n^--r--- 

670] b and 6701a m s v h c l I . ^ ' 

a desc ioed dcv> t tr ^ - dt t j j i 

button 7503 to assign a convergence angle So the optical systems on in the ■imit value 6, M and to paraHefly displace 
the Optical systems when the convergence angle has reached m® lirrt* value 9 LM . in this manner, the user can freesy 
adjust &0 image ; \ / with different para* lax to his or har tasta" 

TH« operation method of the image sensing opt-cai systems 6701b and 6701a am this embodiment is an example, 
and.thfttcortvergdnce ang so trola d pan=;;ei<S;splacement tro can be arbitrarily sea i within tha ranqe of the 
~ " s - J lM o J the con-ergence ang e'o ad usf-e posits of the - I t 

670td and 6701a and to implement the parallax control of the poncipal object 6504 

m me 15th to 1 7th embodiments described above, the automatic paraifax control :n two. right and ten images has 
N "a -m - " r m m K w «di --tt d< ^ i t ^ m 

7 " % s e I 7h mam parallel o;sf nt cor OCDs 

6703, a N " so example of t g - pticaf systems 0701b a i, the centre 

method using the control button 7503 has been exemplified. Alternatives', any other devices may be used as iong as 
they seme as user interfaces tt itrci t ' t ( v xr 

<Stote Application Prograrm- 

As m= 5. i m i * i urn " " i . -< i ^ t 

v. bji , -'lA , " , t'*no ffl c t-g; 5" *c c o 

^ 1" i " - 1 m i j n - !n = u e - ^ n e - ' ■>* 



The thesis module" is a nodule > t f syn t sis method g ng a singte 

synthesized image fiom two. light and lent Images sensed by two. right and left image sensing optical systems. The 
"switching models-" is a program module for switching the plurality of synthesis methods. 

The f he a u if -> s r > - ho Is r 1 - - ? r , 

priority to the synthesis speed, and m second method of the plurality of synthase methods is synthesis method 2 
synthesizing images while giving pnonty to the image quality of the synthesized image In synthesis; method 1. two. 
right and left sensed images 3 t h rj 

two. right and left sensed images ore sob;edted to corrections of, e g., ngnt-and-iert differences of luminance levels 

na r n 3 pping regie > s r hi and left image 

detected and the mages are syniheaized uaingthe overlapping amount calculated based on the detected ove- appa > 
sgior „f-ermc-e f^c sv. tenrg mccufe* ssiscts svntnasts meroc 1 r the th tdugr> d-splay mode 3^d soiects 
synthas e method 2 in the recording and reproduction modes. 

Anoton scrape >~ec -m tha stores s program fo confro ng ^a above rc-ferod b -*cJ-i mage sensinc r 
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oaratus can store a program cede of at least a "control module", asshownfin Fig. 38. 

The'cor I mo e eaer ; -n mcduse 'or ccn j Q aa>si the parallax of the principal object selectee from 
( h. ? pt r "\ 'cri" ;i r p * f n 3 e " r s 

system. When the cenveraence angle has reached the limit value, the "control modeie" paraileily displaces the image 
t c ) X r n f | t*a ~ " i f 1 

re-n „ * «rrs a I p h - < j area n' me, r r, -ot 

m vaj- re ~rh-„ 1 » * ^ c k " 1 1 1 

i_ 1 j t 1 " h 

„_ n v «- f f n f o o < r 

tjc . . r •> -V ^ v a<m t - ^ m -v ( 

t r 19 o 3m 1 1 -ten s o ne *<o na sing opticas syst ?nse image by adjusting trtt 

i > j f - i n - ^ f 



t r - lus and v 



il system has reached me limit v 



tl ~C v J i 1 -i> v, ; A 

•5 j. 1i- -1 hep ai object in the mage 

i ht mo ° stiiS another sioraqe medium that stares a program tar controlling ths above-mentioned binocular 
image n ty apparatus can store program codes of a it value setting module' and 'control moduie*. as; shown 
in: Fig, S9. 

The "limit veins setting module* Is a program module for setting the limit value of tne convergence angle of eacn 
20 jmage sensing optical system. Tne "control module" r a program module lor controlling the image sensing optical 
systems by controlling the convergence ancle or parailet-dispiacement amouni when the convergence angle is equal 
to o 3i- I ^ r tna - f v ^ . set yy the r t .'a ue n noo 1 e and ec- >oi i ig 1 1 ^a< e s g -p s 
s y st 3ms Oj con • - icement amount when the convergence angle has reached ths limit value. 

25 <Other VI < 

The invention that is disclosed in the binocular imaging systems according to the various embodiments above also 
;at-m t. - eh- ^ i ^ n m- a\f i"-m n - it on scorned 

As many apDc e it *>de!y different embodim preser c l arti fro 

: so the spirit and scope thereof, it Is to be understood that the invention is not limited to the specific embodiments thereof 
except as defined in the appended claims. 



Ciaims 

3$ 

t, A multi-eye came; a which comonses a plurality of image sensing means for sensing en Image e! an object, com- 
prising: 

firs; synthesis means (36) for synthesizing a plurality of image signals of the object sensed by sad plurality 
40 of image sensing means, and 

display cleans for displaying an image signal synthesized ny said t r synthesis means. 

imace signals of tha object sensed by said plurality of image sensing means. 10 a nec<iimsns;onai image signal. 
4$ sac n, r r V S u d <~ " ~ s 

moans. 

The camera accord g to claim 2, fur oomp g first selection m 43} f It l 
and second synthesis means 

so 

4. Tne camera according to claim 3, wherein saie first synmes s means synthesizes tne plurality of -nags signals to 
? t ,ee r- r h -> - " t ^ j " d o s .ay 

j t at ~ 5 ji* 1 

ss s c ™ q t^ -1 erem sa d a sDisy means corrp - a ^- h i' % r j v f f 

setlaca thereof. 



6, The camera according to claim i wherein said dlsesay means comprises a detachable parallax barrier wrth respect 



£P 0 830 034 A1 



to an observer, 

?. m -it-- n I ■:' •:: ~c s ~ sn~ a \ b t s i ^, m so; 

signal placing spectacles with shutters to operate in synchronism with a display period of the tVKXfemensions) 
image signal. 

6. v_tr r oi said pluraiit 

Image sensing means, ths image or _ obiec; being sensed by said one image sensing means selected by said 
second seieeiicn means 

v o m i i n t is ■ o tc fc * "i sspsct 

n t 2 camera furth com es pivot angle \ r leans fo tecting p tangles 

ot ? d 

10. The cams n - eg claim 9, further come i jrd selectic e r fc - v'8i> actings rstand 
second synthesis means on the basis of the pivot angles detected by said pivot angle detection means 

11. An m s proe sing net iod c c . g 

Sha irrapc so i g eye:: a ol i - r 

m- <■ mt , - it i r " : i t:: r -t) " imj= g = > p <■ -s i tr- - qt >- 0 -g 

the display step of displaying the tmse-dirsensioriaf image signai synthesized in the first synthesis step. 

12. The method according to ciaim It, further comprising the second synthesis step cf synthesizing the plurality of 
.mage signals of ths object sensed m the image sensing step to a two-dimensional -image signal, the display step 

i % m , - i - 5c= - mthes s *tep 



14. ths method according to claim it, wherein a display period of the fhiee-dimensionai image signal in the display 
step is set tc bs twice a display period o! the two-dimensiona! image signal. 

15. An imar^ s - usccmorSing 

3 piuraiity c nags sensing ms s 

output means for out putting a plurality o ; image signals obtained by said plurality ol image sensing means and 
a synchronization signal synchronized ai each of f plurality c! image signals. 

18. Ths apparatus according to ciairn 15, further comprising display means tor display ma the plurality ol image j a j 
so . c l ml r a - a - - , < - r r r 4 j - , u air 

7. ^i' 3f.' <. r l-r <■ - q a \ ti i mt r ~ Jar oe d I 

vision signal. 

18, ~"o >~ t j i f •• i " r - i j c r 

display means of a computer. 

19, The apparatus according tc claim >5, wherein sad output means alternately outputs the plurality of image signals 
on a time axis. 

20, Tee apparatus according to claim 15. wherein said output means cutouts ths plurality of image signals so mat 
individual images are arranged on a display screen of tiaasmiimensioriai display means cf a oomDUter 

21, Tr-e apparatus according to claim 15, further comprising: 

mode sefling means for selectively setting ens cf a i ° una image mode for three-dimensbnally 
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displaying tns plurality of Image signals, and a two-dimensional image mode for synthesizing the plurality of 
>td - i - v us,- - ?ea tm= 3 - j - 3 

j 3 ,cr- r • -j... .: - r u th ^n rradt -~ 

- -i tiiac :.r ; t mea 5 

22. T ecpo ig to claim 15. further con ft ns or ng aid put mea and 

irssc ompi t r 

23. A t r oca J 1 5 ng 

a plurality of image input means; 

input means tor inputting a three-dtmensicrai : msos signal from an external device; and 

„ f - ^ ,i - j f - J 1f in 1 

, j ( rn„r- e -~ i l 1 rr- r uf 1 ct means. 

24. T^e appa > r - t r t ms age 1 
dimensional image data. 

25. The app j to claim 2% wherein the mrts dnre^ c^ r-aq* ,tc ' -u 0 ta for generating 
a three-dimensional image. 

26. The apparatus according to claim 23, wherein ssid plurality of image input means respectively comprise image 
sensing means. 

27. The apparatus according to claim 23, herein a! 'feast one of said plurality of Imags input means comprises an 
adapter device for inputting a three-dimensional image signal. 

> b c - " X rm t ^ w 0 d J 

nputframsaldpi in. t and s r - o c a or m each of the plurality 

of image signals. 

29. An image sensing method comprising: 

c ^ i , r r 0 me node and 

ins step of synthesizing two, right and left images sensed by two. nor t and Is 3s ssr 3 optical systems 
;oa single synthesized image - accordance with tns selected image synthesis method. 

3C Tv = 5 ~ < 1 r + " " " ' " " ' t 1 =fd n 

x \ r - i- o H -1 •= " i n~, fc kx; tor synthesizing 

nagas white giving orient to image c alityc des- thesizsd image 

31. The method according to ctaim 30, wherein the first synthesis method synthesizes the two, right and left sensed 
f -i ] - mt 

> ^\ro 1 tie 'n«V-r t Jr t i - ^~ rt -it 

? o -1. d l. -msr r - 

i-.vc images, and synthesizes the two images using an overlapping amount calculated eased on the overlapping 
region. 

32 n g 9. wherein the sele tap includes the step oi 2 j, 

L U JU ° """-^ S 

33. The method according to claim 29. wherein the synthesized image is a panoramic synthesized image. 

34. The method according to claim 29. wherein the synthesized image is an image tor stereoscopic view 

35. An image sensing apparatus comprising. 

synthesis means having a plurality of synthesis methods for generating, a single synthesized Image from two 
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i - and isft images sensed by two. right and M image sens p ; stems; and 
i i i ^th ds 

36 if >5. v. plufsfity of synthesis m s method 

for synthesizing !hs images white giving priority to a synthesis spesd : land a second synthesis matbccf tor symthe- 
" " " 2 A ' i - 3 " t h t „ i "aqc 

37, The apparatus according to dam 35 wherein she first synthesis mo-hod synthesizes the two, right and ieft sensed 
mages by giving a predetermined ove socmg am ml and ins second synthesis method corrects ghhsod-iefi 
differences of luminance ieveis and color informs- ion. and trapezoidal distortions of this two, right and ieft sensed 
■mages, detects an overlapping region between the two images. => syrtihesirms tne two images using an over- 
lapping amount calculated based on the overlapping region. 

38. Theap r - iing o claim 35 here ichi < s e _ - 1 s nthesis method in a 8 gh 

J selects scene sis method m ^ p r 

33. The apparatus according to damn 35, wherem tne synthesized image is a panoramic synthesized image, 

40. The apparatus according to claim 35, whamm ths synthesoed Image is an image for stereoscopic view 

41. An image sensing method comprising; 

the step of sensing a pair or Images having parallax using two image sensing optical systems; and 

this control step of controlling to adjust parallax of a pair of partial images of a principal object selected from 

the pair of sensed images. 

42. The method according to claim 41 , wherein the con-roi step includes the step of setting a limit value of a conver- 
ger e ems 

*3 j aim 41 • esr h.. p inou e step c - retkx of the 

principal object n" tne images by paralleliy displacing an Image to be displayed when the convergence angle of 
each of tne image sensing optical systems has reached the limit value. 

44. Tne method acenrdmg to v si na 1 n -ps n el-d sp acemen ca a rdh ctssp jcs^ ir" image sens g optica 
systems, and tne ccntiol step includes the step of adjusting the oamiiax or the principal object m the Images by 

w -m a bass distance between h mnage sensing optica- systems to sense Images when the convergence 
angle ot each of the image sensing optical systems has reached the limit value 

45. The method according to claim 41. wherein the paralfehdfsciacsmsnt paralleffy displaces image sensing devices 
in the image sensing optical systems, and the control step includes the step ot adjusting the parallax of the principal 
ocjeet in the images oy paralleliy displacing the image sensing devices in the Image sensing optica! systems to 
<=«0 ate ( t t n n 1 
each of the Image sensing optical systems has reached ths limit value 

4€ The method according to claim 4i, wherem the c - paralleliy displaces right and left sensed 

iro tne csndol step -m-..oes t - p-roj n t« m-gec 

peraiieiiy displacing the right and left sensed images to generals an image 'or stei'eoscopic view when the con- 
I t, t =■ -r~ - - * > r -r 

47,. An image sensing method comprising: 

" slop jf sens s a pair r - s c , g paraits> using * n ra~~ - sing op Oil sys ams - c 

a -vnt - _ 4 1 r 1 

controlling the image sensing optica} systems by controlling the convergence angle or a parcel-displacement 
amount of ths image sensing optical, systems- v&en-tfiie cdftveitjjeScte. angle & -not- more than the limit value, 
nc n f'tint^eimacese^SingoptraiSyStemsbyconfO'ing^heo^^iiel-dspface^em^r ntofth mace 
sensing optica! systems when the convergence angle has reached the limit value 
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4S. An Image sensing apparatus comprising: 

twc , - c sts -ns for sens'ng a o i > iax -nd 

adjustment means for adjusting the parallax of a principal object selected from ins sensed images. 

s 

49. The .apparatus according to claim 4S. whs ram said adjustmeni means sets a limit value of a convergence angle 
of each of sa;ct image sensing optical systems. 

50. The appanatu c rr 1 adj r r e principa object 
?s in the images by pataiielly displacing an image to be displayed ween the convergence angls of each ct said image 

n • value. 

51. Tne apparatus according to claim 4g. wharesn me parsJieiaPsplaeement indicates parallel-displacement of said 

c"sw& x>\ a ^ytt r 5 ' ' - y < lir*ct hepnnr 

m by shortening 3 bass distance between a image sensing optical systems to sense r <\ • -' m " tne convergence 

ar 3 ! of ea rjf sa;d =ma > s - - an a hed h mit value 

52. The apparatus according to ciaim -■" wherein the m j p ' indicates parallel-displacement of image 

] ^ f -a v. si ti 1 » pm it ax ct 

so principal object In the images by carafleily disslacing the image sensing devices in said image sensing optical 

systems to separate from centers or said two imags sensing optica! systems to sense images whan the conver- 
gence angls of each of said image sensing optical systems has reached the limit value. 

53. The p men', ot right 
and left sensed images, and said adjustment means adjusts the parallax of the principal object in the Images by 
parailsily displacing the right and n - - > « n the.. con- 
veyance angle o; aaeft ot saw image sensing optica; systems has reached I l value. 

54. An maps- - tram at m t --^ct at 

as 

two image sensing optica! systems for sensing a pair of images having parallax; 

im ill pj mmgm-^-rsfo set nga itmtrva uem s jan ~i 

control means for controlling said image sensing optical systems by controlling the convergence angle or a 
3S parallel onla n m aJr ^ t c e^ <- " _ r~ = i 

man the limit I etting me iimg sale im < systs 

onvergance 

angle has reached the limit value, 

4Q 55 =■ , it r? > f. < ? s -d w 

■re ^ s medium t tores a pro ' f < ar rand cor mags sensing apparat 

comprising: 

*s synthesis program code means for describing a plurality of different synthesis methods for generating a single 

synthesized Image from two, right and iett images sensed by two ngh; and 1st; image sensing optical systems, 
and 

m:r;-r ma r i t o mo 

$6 5' ~~ ' -i a ^ - ' a 3 - 1 - ' ' * i a- r r 

sw^es ^t r n i hi- - h > 

images while giving priority to image quality of the synthesized image 

58, The medium according to csaim 56 wherein the first synthesis method synthesizes tne two, right and left sensed 
as images by giving a predetermined overlapping amount, and 

the second synthesis memoes corrects right-and-left differences of luminance levels and color information, 
and trapecoidai distortions ot tne Swo, right and left sensed images, detects an overlapping region between the 
two Images, and synthesizes the two images using an overlapping amount calculated eased on the overlapping 
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59, The m ad t 5 » 1 p i 

synthesis method in a through dsspiay mods and selects a program of she seconc synthesis method in recording 
and reproduction modes 

SO ~ o - 

81. The medium according to claim 56, wherein the syrahesised image is an image for stereoscopic view 

82. A storage = i which is executed -> ( - 1 - image sensing apparatus, 
comprising: 

r i , - i ror r 

second progtam code means for con-rolling to adjust the parallax of - principal object selected from -he sensed 
images. 

S? f M 1 ro " I ,r - 1- 1 ii - - 5 -o, n 

angle of each o ; the image sensing f systems. 

64. The medium according to claim 52, wherein said second program coca means: adjusts the parallax of the principal 
object in ths images by paraBeily displacing an image t . f 92 5h of me 

image sensing opticas systems hss reached a \rtM value. 

83. ths medium according to claim 62. wherein when the parallei-disp a - sm - -t n riicates parallel-displacement of ths 
image sor * « 3 op . , - -s> -u se and nog. -~ cods means adjusts the parallax of the principal object: in 
the images by shortening a base distance between ths image sensing optica! systems to sense images when a 

/erg ngl each s Ima - pheai systems has reaches 1 it. js 

S8. The medium according to claim 82, wherein whan ' - nent indicates parallel-displacement of 

image sensing devices in ths Image sensing optica! systems., said second program code means adjusts ths oaraifax 
of the principal object r rhe - roc; by paraiislly 0 ro .one the image sensing devices in the image sensing optical 

-ages when a conve pu ->ce 

angle of each ot ths image sensing optica) systems has maenad a iimit value. 

87. The apparatus according to claim 5m, wherein when the paraiial-dispiacsmsnt indicates oaraiiel-oispiacemant of 
right and left sensed images, saic second program code means adjusts the saraiiax of the p.uncspa; object in the 
images by paraiially displacing the right and left sensed images to generate an image for stereoscopic view when 
a convergence sngie of each of the image sensing optica! sysroms has reached a hmit value. 

66 arags madly ss a program wj- sax Jtedbyacomput ro m s mage sin gap pa rates 

for sensing a pan of images having parallax using two Image sensing optica! systems, comprising. 

imsf program code means for setting a limit value of 3 convergence angle of each of the image sensing ooiloai 
systems; and 

cond prog ems Py control! gence 

angk u^ro ro - - - h 3 her variance ingle 

is not more than tee ! vahro set by sam hrs; program coda means, and controlling tee image sensing optical 
systems by controiimg tne parailei-cispi&eemem amcom of the Image sensing optical systems whan the con- 
vergence angle has reached the limit value, 

69. A r method n w-jch 3 - - nub eyo carre'a oe " our ij 
h j j - ta cefimrg a synthsslsed image of the ot 

70. An Image processing apparatus or method In which image data defining a pie-ratty of Images ot an obiect is proc- 
essed to generate fnree-dsmansiona! data of tne object for display. 
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71. A i mag<- - rstus or trained in when threes r a fo an ob^c; ana m^ps s p-ais are se- 
leciiveiv *spiayeti. 

72. A? image prccsssrg apparatus or method n A-hich a syndesis msthod s st jcted f cm a p jrslit) ■" avaiteble 
synthesis methods and used to process; image data defining "right" and left' image? of an object to generate 
image data deiinmg a sym.hesm.ad image. 

73. An image Drocmssing apparatus or method in which image data denning first and second images with parallax is 
processed tc adjust the paraiiax for an object in the images. 

74. An apparatus ar method in which image sensing optical systems tor producing paraiiax imager; of an object are 

-<cof 
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